Page History

Table of Contents

Introduction

The Fedora Community has raised significant concerns about retrieval times for resources with many members. By "members" we are talking about at least three scenarios

...

Using a set of tests originally devised by Esmé Cowles I modified them slightly and added a couple of new tests. The latest versions can be found here : test scripts. The test results, in most cases, reflect that fastest retrieval speeds I was able to get. Initial retrieval times tended to be slower in most cases due to the fact that modeshape needs to warm up the cache. The scripts now make an initial GET call to warm the cache before timing a second call to the same resource.

...

Install postgresql
Install mysql-server
Build the fcrepo4 project from the specified commit.
Run fcrepo-webapp using mvn clean jetty:run and the appropriate MAVEN_OPTS such as fcrepo.modeshape.configuration, fcrepo.mysql.username, fcrepo.mysql.password, fcrepo.postgres.username, fcrepo.postgres.password.
git clone https://github.com/dbernsteinfcrepo4-labs/fcrepo-performance-test-testsscripts.git
For each test, I ran the scripts with 1000 and 10000 items.

Instance Definition

AWS / Ubuntu 16 / Oracle Java 8 / m3.medium (3.7 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 1)
Lenovo / Ubuntu 16.10 / Java HotSpot 1.8.0_111 / 11.6 GiB memory / Intel i7-4600U CPU @ 2.10GHz x 4
AWS / Ubuntu 16 / Oracle Java 8 / m3.xlarge (14 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 4)
AWS / Ubuntu 16 / Oracle Java 8 / c4.xlarge (7.5 GiB memory / Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 4)
AWS / Ubuntu 16 / Oracle Java 8 / c4.2xlarge (15 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 8)
AWS / Ubuntu 16 / Oracle Java 8 / c4.4xlarge (30 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 16)

n-children.sh

FCREPO Version	Repo	Commit	Environment	Modeshape Config	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	dbernstein	b60d4e	5	file-simple	10000	5.222	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	daa11f3	5	file-simple	10000	5.567	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	b60d4e	6	file-simple	10000	4.859	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	daa11f3	6	file-simple	10000	4.841	Danny Bernstein

n-members.sh

...

Number of relations: 1000

FCREPO VersionRepobranchCommitmodeshape configEnvironment# of relations

Test Duration

(seconds)

TesterNotes4.8.0-SNAPSHOTfcrepo4master2df32file-simple110006.570

Danny Bernstein

4.7.1fcrepo4master

FCREPO Version	Repo	Branch	Commit	modeshape	Environment	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	1000	1.408	Danny Bernstein
4.7.1	fcrepo4master	4.7.1	546f5a5	file-simple	2	1000	1.45	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	10,000	28.583	Danny Bernstein
4.7.1	fcrepo4master	4.7.1	546f5a5	file-simple	2	10,000	24.79	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e	file-simple	3	10,000	9.58	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	3	10,000	12.16	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	4	10,000	9.71	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	5	10,000	8.51	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT			b60d4e	file-simple	5	10,000	8.815	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	6	10,000	8.29	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT			b60d4e	file-simple	6	10,000	8.661	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	1	1000	1.543	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	110000	10,000	61.381	Danny Bernstein	perhaps postgres needs caching configured?
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	3	1000	0.592	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	310000	10,000	39.671	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	3	1000	4.435	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	310000	10,000	39.486	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	f453a	jdbc-postgresql	3	1000	0.561	Danny Bernstein	1 million item cache
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	f453a	jdbc-postgresql	310000	10,000	5.334	Danny Bernstein	1 million item cache
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	jdbc-postgresql	3	1000	0.633	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	jdbc-postgresql	310000	10,000	5.398	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	mysql-postgresql	3	1000	0.820	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	mysql-postgresql	310000	10,000	7.726	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-postgresql-s3	3	1000	0.701	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-postgresql-s3	310000	10,000	5.485	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysql-s3	3	1000	0.864	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysql-s3	310000	10,000	7.395	Danny Bernstein	1 million item cache + parallelized

...

4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000	13.07	Andrew Woods	cacheSize = 50,000
4.7.1	fcrepo4	4.7.1

546f5a5

file-simple

2

1000

10,000

2

10.

80

30

Andrew Woods

cacheSize = 1,000,000

n-memberof.sh

Number of relations: 1000

4.8.0000

FCREPO Version	Repo	branch	Commit	modeshape config	Environment	# of relations	Test Duration (seconds)	Tester	Notes
4.8.0	-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	10,000100086	6.	570	Danny Bernstein
4.7.1	fcrepo4master	4.7.1	546f5a5	file-simple	2	10,0001000	572.3580	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e2df32	file-simple	31	10,000	3086.02000	Danny Bernstein	Unlike the n-member example, results begin streaming right away - so the response begins streaming within 2 seconds.
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000	57.35	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e	4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	3	10,000	2930.97502	Danny Bernstein parallel	streams enabledUnlike the n-member example, results begin streaming right away - so the response begins streaming within 2 seconds.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	43	10,000	29.975	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	4	10,000	24.790	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	5	10,000	20.992	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e	file-simple	5	10,000	26.357	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	6	10,000	20.337	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e	file-simple	6	10,000	25.782	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	1	1000	11.414	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	110000	10,000	194	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	crepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	3	1000	3.961	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	crepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	310000	10,000	53.452	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	3	1000	10.833	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	310000	10,000	109.530	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	f453a	jdbc-postgresql	3	10000	22.963	Danny Bernstein	1 million item cache.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	5138b4	jdbc-postgresql	310000	10,000	11.559	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	5138b4	mysql-postgresql	3	1000	2.337	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	5138b4	mysql-postgresql	310000	10,000	14.998	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysqlpostgresql-s3	3	1000	21.374883	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysqlpostgresql-s3	3	10000	14.956	10,000		Danny Bernstein	1 million item cache + parallelized

...

FCREPO VersionCommitEnvironment# of relations

Test Duration

(seconds)

4.8.0-SNAPSHOT

2df32

dbernstein

1

fcrepo-2402

10000.039

75dd1

jdbc-mysql-s3

3

1000

2.374

Danny Bernstein

4.7.1546f5a5210000.14

Andrew Woods

1 million item cache + parallelized

4.8.0-SNAPSHOT

2df321

dbernstein

fcrepo-2402

75dd1

jdbc-mysql-s3

3

10,000

0

14.

063

956	Danny Bernstein

4

1 million item cache + parallelized
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000

0

31.

17

91	Andrew Woods

n-properties.sh

cacheSize = 50,000
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000	23.07	Andrew Woods	cacheSize = 1,000,000

n-uris.sh

Tester 1000

FCREPO Version	Commit	Environment	# of relations	Test Duration (seconds)	FCREPO Version
4.8.0-SNAPSHOT	2df32	1	1000	0.060039	Danny Bernstein
4.7.1	546f5a5	2	1000	0.08914	Andrew Woods
4.8.0-SNAPSHOT	2df32	1	10,000	0.119063	Danny Bernstein
4.7.1	546f5a5	2	10,000	0.28117	Andrew Woods

n-binaries.sh (time for loading binary files)

n-properties.sh

FCREPO Version	Commit	Environment	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	2df32	1	1000	0.060	Danny Bernstein
4.7.1	546f5a5	2	1000	0.089	Andrew Woods
4.8.0-SNAPSHOT	2df32	1	10,000	0.119	Danny Bernstein
4.7.1	546f5a5	2	10,000	0.281	Andrew Woods

n-binaries.sh (time for loading binary files)

repo

branch

Commit

Environment

Modeshape Config

# of binaries

size in KB

Test Duration

(seconds)

Tester

https://github.com/bbranan/fcrepo4.git

fcrepo-2402

f0a51e

1

file-s3

1000

00:06:02

Danny Bernstein

https://github.com/bbranan/fcrepo4.git

fcrepo-2402

f0a51e

1

file-simple

1000

00:02:16

Danny Bernstein

repobranchCommitEnvironmentModeshape Config# of binariessize in KB

Test Duration

(seconds)

Tester


https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e	1

file

jdbc-

s3

postgresql

1000

00:

06

02:

02

13	Danny Bernstein
https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e

1

3

file

jdbc-postgresql-

simple

s3

1000

00:

02

06:

16

36	Danny Bernstein
https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e

1

3	jdbc-mysql-

postgresql

s3

1000

00:

02

06:

13

32	Danny Bernstein

Page tree

Versions Compared

Old Version 61

New Version Current

Key

Introduction

Instance Definition

n-children.sh

n-members.sh

n-uris.sh

n-binaries.sh (time for loading binary files)

n-binaries.sh (time for loading binary files)

Conclusions