Page History

...

Using a set of tests originally devised by Esmé Cowles I modified them slightly and added a couple of new tests. The latest versions can be found here : test scripts. The test results, in most cases, reflect that fastest retrieval speeds I was able to get. Initial retrieval times tended to be slower in most cases due to the fact that modeshape needs to warm up the cache. The scripts now make an initial GET call to warm the cache before timing a second call to the same resource.

...

Install postgresql
Install mysql-server
Build the fcrepo4 project from the specified commit.
Run fcrepo-webapp using mvn clean jetty:run and the appropriate MAVEN_OPTS such as fcrepo.modeshape.configuration, fcrepo.mysql.username, fcrepo.mysql.password, fcrepo.postgres.username, fcrepo.postgres.password.
git clone https://github.com/dbernsteinfcrepo4-labs/fcrepo-performance-test-testsscripts.git
For each test, I ran the scripts with 1000 and 10000 items.

Instance Definition

AWS / Ubuntu 16 / Oracle Java 8 / m3.medium (3.7 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 1)
Lenovo / Ubuntu 16.10 / Java HotSpot 1.8.0_111 / 11.6 GiB memory / Intel i7-4600U CPU @ 2.10GHz x 4
AWS / Ubuntu 16 / Oracle Java 8 / m3.xlarge (14 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 4)
AWS / Ubuntu 16 / Oracle Java 8 / c4.xlarge (7.5 GiB memory / Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 4)
AWS / Ubuntu 16 / Oracle Java 8 / c4.2xlarge (15 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 8)
AWS / Ubuntu 16 / Oracle Java 8 / c4.4xlarge (30 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 16)

n-children.sh

FCREPO Version	Repo	Commit	Environment	Modeshape Config	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	dbernstein	b60d4e	5	file-simple	10000	5.222	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	daa11f3	5	file-simple	10000	5.567	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	b60d4e	6	file-simple	10000	4.859	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	daa11f3	6	file-simple	10000	4.841	Danny Bernstein

n-members.sh

...

Number of relations: 1000

FCREPO VersionRepobranchCommitmodeshape configEnvironment# of relations

Test Duration

(seconds)

TesterNotes4.8.0-SNAPSHOTfcrepo4master2df32file-simple110006.570

Danny Bernstein

FCREPO Version	Repo	Branch	Commit	modeshape	Environment	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	1000	1.408	Danny Bernstein
4.7.1	fcrepo4master	4.7.1	546f5a5	file-simple	2	1000	1.45	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	10,000	28.583	Danny Bernstein
4.7.1	fcrepo4master	4.7.1	546f5a5	file-simple	2	10,000	24.79	Andrew Woods
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e	file-simple	3	10,000	9.58	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	3	10,000	12.16	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	4	10,000	9.71	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	5	10,000	8.51	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT			b60d4e	file-simple	5	10,000	8.815	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	6	10,000	8.29	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT			b60d4e	file-simple	6	10,000	8.661	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	1	1000	1.543	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	110000	10,000	61.381	Danny Bernstein	perhaps postgres needs caching configured?
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	3	1000	0.592	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	310000	10,000	39.671	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	3	1000	4.435	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	310000	10,000	39.486	Danny Bernstein
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	f453a	jdbc-postgresql	3	1000	0.561	Danny Bernstein	1 million item cache
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	f453a	jdbc-postgresql	310000	10,000	5.334	Danny Bernstein	1 million item cache
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	jdbc-postgresql	3	1000	0.633	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	jdbc-postgresql	310000	10,000	5.398	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	mysql-postgresql	3	1000	0.820	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-large-cache	5138b4	mysql-postgresql	310000	10,000	7.726	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-postgresql-s3	3	1000	0.701	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-postgresql-s3	310000	10,000	5.485	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysql-s3	3	1000	0.864	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysql-s3	310000	10,000	7.395	Danny Bernstein	1 million item cache + parallelized

...

4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000	13.07	Andrew Woods	cacheSize = 50,000
4.7.1	fcrepo4

4.7.1

fcrepo4master

546f5a5

file-simple

2

1000

10,000

2

10.

80

30

Andrew Woods

cacheSize = 1,000,000

n-memberof.sh

Number of relations: 1000

Andrew Woods0224790parallel streams enabled.

FCREPO Version	Repo	branch	Commit	modeshape config	Environment	# of relations	Test Duration (seconds)	Tester	Notes	4.8.0-SNAPSHOT	fcrepo4	master	2df32	file-simple	1	10,000	86.000	Danny Bernstein	4.7.1	fcrepo4	master	546f5a5	file-simple	2	10,000	57.35
4.8.0-SNAPSHOT	fcrepo4	master	b60d4e2df32	file-simple	31	10,000100030	6.	570	Danny Bernstein Unlike the n-member example, results begin streaming right away - so the response begins streaming within 2 seconds.
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	1000	2.80	Andrew Woods
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	fcrepo4	master	2df32daa11f3	file-simple	31	10,000	2986.975000	Danny Bernstein	parallel streams enabled.
4.7.1	fcrepo4	4.87.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	1	546f5a5	file-simple4	2	10,000	57.	Danny Bernstein	35	Andrew Woods
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	fcrepo4	master	b60d4edaa11f3	file-simple	53	10,000	2030.99202	Danny Bernstein parallel	streams enabledUnlike the n-member example, results begin streaming right away - so the response begins streaming within 2 seconds.
4.8.0-SNAPSHOT	fcrepo4	master	dbernstein	fcrepo-2105-v4-parallelization	daa11f3b60d4e	file-simple	53	10,000	2629.357975	Danny Bernstein	parallel streams enabled.
44.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-parallelization	daa11f3	file-simple	64	10,000	2024.337790	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	fcrepo4	master	dbernstein	fcrepo-2105-v4-parallelization	daa11f3b60d4e	file-simple	65	10,000	2520.782992	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	bbrananfcrepo4	fcrepo-2402master	f0a51eb60d4e	jdbcfile-postgresqlsimple	151000	10,000	1126.414357	Danny Bernstein
4.8.0-SNAPSHOT	bbranandbernstein	fcrepo-2402	f0a51e	jdbc-postgresql	1	10000	2105-v4-parallelization	daa11f3	file-simple	6	10,000	20.337194	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	crepo-2105-v4-parallelization	daa11f3	jdbc-postgresql	3	1000	fcrepo4	master	b60d4e	file-simple	6	10,000	25.7823.961	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernsteinbbranan	crepo-2105-v4-parallelization	fcrepo-2402	f0a51edaa11f3	jdbc-postgresql	31	100001000	5311.452414	Danny Bernstein parallel streams enabled.
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	f0a51e	jdbc-postgresql	31	1000	10,000	19410.833	Danny Bernstein
4.8.0-SNAPSHOT	bbranan	fcrepo-2402	dbernstein	crepo-2105-v4-parallelization	daa11f3f0a51e	jdbc-postgresql	3	100001000	1093.530961	Danny Bernstein	parallel streams enabled.
4.8.0-SNAPSHOT	dbernstein	fcrepocrepo-2105-largev4-cacheparallelization	f453adaa11f3	jdbc-postgresql	310000	10,000	2253.963452	Danny Bernstein	1 million item cacheparallel streams enabled.
4.8.0-SNAPSHOT	dbernsteinbbranan	fcrepo-2105-v4-parallelization2402	f0a51e5138b4	jdbc-postgresql	3	100001000	1110.559833	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernsteinbbranan	fcrepo-2105-v4-parallelization	5138b4	2402	f0a51e	jdbcmysql-postgresql	31000	10,000	2109.337530	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4large-parallelizationcache	5138b4f453a	mysqljdbc-postgresql	3	10000	1422.998963	Danny Bernstein	1 million item cache + parallelized.
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-2402parallelization	75dd15138b4	jdbc-postgresql-s3	31000	10,000	111.883559	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2105-v4-2402parallelization	75dd15138b4	jdbcmysql-postgresql-s3	3	100010000	2.337	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	2105-v4-parallelization	5138b4	jdbc-mysql-s3postgresql	31000	10,000	214.374998	Danny Bernstein	1 million item cache + parallelized
4.8.0-SNAPSHOT	dbernstein	fcrepo-2402	75dd1	jdbc-mysqlpostgresql-s3	3	100001000	141.956883	Danny Bernstein	1 million item cache + parallelized

n-uris.sh

4.8.0-SNAPSHOT

dbernstein

fcrepo-2402

75dd1

jdbc-postgresql-s3

3

10,000

Danny Bernstein

1 million item cache + parallelized

FCREPO VersionCommitEnvironment# of relations

Test Duration

(seconds)

4.8.0-SNAPSHOT

2df32

dbernstein

fcrepo-2402

75dd1

jdbc-mysql-s3

3

1

1000

0

2.

039

374	Danny Bernstein	1 million item cache + parallelized
4.

7

8.

1

0-SNAPSHOT

546f5a5

dbernstein

24.8.0-SNAPSHOT2df321

fcrepo-2402

10000.14

Andrew Woods

75dd1

jdbc-mysql-s3

3

10,000

0

14.

063

956	Danny Bernstein	1 million item cache + parallelized
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000

0

31.

17

91	Andrew Woods

...

cacheSize = 50,000
4.7.1	fcrepo4	4.7.1	546f5a5	file-simple	2	10,000	23.07	Andrew Woods	cacheSize = 1,000,000

n-uris.sh

1000

FCREPO Version	Commit	Environment	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	2df32	1	1000	0.060039	Danny Bernstein
4.7.1	546f5a5	2	1000	0.08914	Andrew Woods
4.8.0-SNAPSHOT	2df32	1	10,000	0.119063	Danny Bernstein
4.7.1	546f5a5	2	10,000	0.28117	Andrew Woods

n-binaries.sh (time for loading binary files)

n-properties.sh

FCREPO Version	Commit	Environment	# of relations	Test Duration (seconds)	Tester
4.8.0-SNAPSHOT	2df32	1	1000	0.060	Danny Bernstein
4.7.1	546f5a5	2	1000	0.089	Andrew Woods
4.8.0-SNAPSHOT	2df32	1	10,000	0.119	Danny Bernstein
4.7.1	546f5a5	2	10,000	0.281	Andrew Woods

n-binaries.sh (time for loading binary files)

repo

branch

Commit

Environment

Modeshape Config

# of binaries

size in KB

Test Duration

(seconds)

Tester

https://github.com/bbranan/fcrepo4.git

fcrepo-2402

f0a51e

1

file-s3

1000

00:06:02

Danny Bernstein

https://github.com/bbranan/fcrepo4.git

fcrepo-2402

f0a51e

1

file-simple

1000

00:02:16

Danny Bernstein

repobranchCommitEnvironmentModeshape Config# of binariessize in KB

Test Duration

(seconds)

Tester


https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e	1

file

jdbc-

s3

postgresql

1000

00:

06

02:

02

13	Danny Bernstein
https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e

1

3

file

jdbc-postgresql-

simple

s3

1000

00:

02

06:

16

36	Danny Bernstein
https://github.com/bbranan/fcrepo4.git	fcrepo-2402	f0a51e

1

3	jdbc-mysql-

postgresql

s3

1000

00:

02

06:

13

32	Danny Bernstein

Page tree

Versions Compared

Old Version 63

New Version Current

Key

Instance Definition

n-children.sh

n-members.sh

n-uris.sh

n-uris.sh

n-binaries.sh (time for loading binary files)

n-binaries.sh (time for loading binary files)

Conclusions