...
Using a set of tests originally devised by Esmé Cowles I modified them slightly and added a couple of new tests. The latest versions can be found here : test scripts. The test results, in most cases, reflect that fastest retrieval speeds I was able to get. Initial retrieval times tended to be slower in most cases due to the fact that modeshape needs to warm up the cache. The scripts now make an initial GET call to warm the cache before timing a second call to the same resource.
...
- Install postgresql
- Install mysql-server
- Build the fcrepo4 project from the specified commit.
- Run fcrepo-webapp using mvn clean jetty:run and the appropriate MAVEN_OPTS such as fcrepo.modeshape.configuration, fcrepo.mysql.username, fcrepo.mysql.password, fcrepo.postgres.username, fcrepo.postgres.password.
- git clone https://github.com/dbernsteinfcrepo4-labs/fcrepo-performance-tests.git
- For each test, I ran the scripts with 1000 and 10000 items.
...
- AWS / Ubuntu 16 / Oracle Java 8 / m3.medium (3.7 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 1)
- Lenovo / Ubuntu 16.10 / Java HotSpot 1.8.0_111 / 11.6 GiB memory / Intel i7-4600U CPU @ 2.10GHz x 4
- AWS / Ubuntu 16 / Oracle Java 8 / m3.xlarge (14 GiB memory / Intel Xeon E5-2670 (Sandy Bridge) Processor @ 2.6 GHz x 4)
- AWS / Ubuntu 16 / Oracle Java 8 / c4.xlarge (7.5 GiB memory / Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 4)
- AWS / Ubuntu 16 / Oracle Java 8 / c4.2xlarge (15 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 8)
- AWS / Ubuntu 16 / Oracle Java 8 / c4.4xlarge (30 GiB memory /Intel Xeon E5-2666 v3 (Haswell) @ 2.9 GHz x 16)
n-children.sh
FCREPO Version | Repo | Branch | Commit | Environment | Modeshape Config | # of relations | Test Duration (seconds) | Tester | |
|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | dbernstein | b60d4e | 5 | file-simple | 10000 | 5.222 | |||
| 4.8.0-SNAPSHOT | dbernstein | daa11f3 | 5 | file-simple | 10000 | 5.567 | |||
| 4.8.0-SNAPSHOT | dbernstein | b60d4e | 6 | file-simple | 10000 | 4.859 | |||
| 4.8.0-SNAPSHOT | dbernstein | daa11f3 | 6 | file-simple | 10000 | 4.841 |
n-members.sh
| FCREPO Version | Repo | Branch | Commit | modeshape | Environment | # of relations | Test Duration (seconds) | Tester | |
|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | file-simple | 1 | 1000 | 1.408 | ||
| 4.7.1 | fcrepo4 | 4.7.1 | 546f5a5 | file-simple | 2 | 1000 | 1.45 | ||
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | file-simple | 1 | 10,000 | 28.583 | ||
| 4.7.1 | fcrepo4 | 4.7.1 | 546f5a5 | file-simple | 2 | 10,000 | 24.79 | ||
| 4.8.0-SNAPSHOT | fcrepo4 | master | b60d4e | file-simple | 3 | 10,000 | 9.58 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 3 | 10,000 | 12.16 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 4 | 10,000 | 9.71 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 5 | 10,000 | 8.51 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | b60d4e | file-simple | 5 | 10,000 | 8.815 | ||||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 6 | 10,000 | 8.29 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | b60d4e | file-simple | 6 | 10,000 | 8.661 | ||||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 1000 | 1.543 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 10,000 | 61.381 | perhaps postgres needs caching configured? | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 1000 | 0.592 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 10,000 | 39.671 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 3 | 1000 | 4.435 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 3 | 10,000 | 39.486 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | f453a | jdbc-postgresql | 3 | 1000 | 0.561 | 1 million item cache | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | f453a | jdbc-postgresql | 3 | 10,000 | 5.334 | 1 million item cache | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | jdbc-postgresql | 3 | 1000 | 0.633 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | jdbc-postgresql | 3 | 10,000 | 5.398 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | mysql-postgresql | 3 | 1000 | 0.820 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | mysql-postgresql | 3 | 10,000 | 7.726 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2402 | 75dd1 | jdbc-postgresql-s3 | 3 | 1000 | 0.701 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2402 | 75dd1 | jdbc-postgresql-s3 | 3 | 10,000 | 5.485 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2402 | 75dd1 | jdbc-mysql-s3 | 3 | 1000 | 0.864 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2402 | 75dd1 | jdbc-mysql-s3 | 3 | 10,000 | 7.395 | 1 million item cache + parallelized | |
| 4.7.1 | fcrepo4 | 4.7.1 | 546f5a5 | file-simple | 2 | 10,000 | 13.07 | cacheSize = 50,000 | |
| 4.7.1 | fcrepo4 | 4.7.1 | 546f5a5 | file-simple | 2 | 10,000 | 10.30 | cacheSize = 1,000,000 |
...
| FCREPO Version | Commit | Environment | # of relations | Test Duration (seconds) | ||
|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | 2df32 | 1 | 1000 | 0.039 | ||
| 4.7.1 | 546f5a5 | 2 | 1000 | 0.14 | ||
| 4.8.0-SNAPSHOT | 2df32 | 1 | 10,000 | 0.063 | ||
| 4.7.1 | 546f5a5 | 2 | 10,000 | 0.17 |
...
| FCREPO Version | Commit | Environment | # of relations | Test Duration (seconds) | Tester | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | 2df32 | 1 | 1000 | 0.060 | |||||||
| 4.7.1 | 546f5a5 | 2 | 1000 | 0.089 | |||||||
| 4.8.0-SNAPSHOT | 2df32 | 1 | 10,000 | 0.119 | |||||||
| 4.7.1 | 546f5a5 | 2 | 10,000 | 0.281 | |||||||
n-binaries.sh (time for loading binary files)
| repo | branch | Commit | Environment | Modeshape Config | # of binaries | size in KB | Test Duration (seconds) | Tester |
|---|---|---|---|---|---|---|---|---|
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | file-s3 | 1000 | 1000 | 00:06:02 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | file-simple | 1000 | 1000 | 00:02:16 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | jdbc-postgresql | 1000 | 1000 | 00:02:13 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 3 | jdbc-postgresql-s3 | 1000 | 1000 | 00:06:36 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 3 | jdbc-mysql-s3 | 1000 | 1000 | 00:06:32 |