| Table of Contents |
|---|
Introduction
The Fedora Community has raised significant concerns about retrieval times for resources with many members. By "members" we are talking about at least three scenarios
- Many children
- Many has_member outlinks
- Many member_of inlinks
Using a set of tests originally devised by Esmé Cowles I modified them slightly and added a couple of new tests. The latest versions can be found here : test scripts.
Setup:
In order to set up the machines for the tests, I performed the following steps.
- Install postgresql
- Install mysql-server
- Build the fcrepo4 project from the specified commit.
- Run fcrepo-webapp using mvn clean jetty:run and the appropriate MAVEN_OPTS such as fcrepo.modeshape.configuration, fcrepo.mysql.username, fcrepo.mysql.password, fcrepo.postgres.username, fcrepo.postgres.password.
- git clone https://github.com/dbernstein/fcrepo-performance-tests.git
Instance Definition
...
n-children.sh
FCREPO Version | Repo | Branch | Commit | Environment | Modeshape Config | # of relations | Test Duration (seconds) | Tester | |
|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | dbernstein | b60d4e | 5 | file-simple | 10000 | 5.222 | |||
| 4.8.0-SNAPSHOT | dbernstein | daa11f3 | 5 | file-simple | 10000 | 5.567 | |||
| 4.8.0-SNAPSHOT | dbernstein | b60d4e | 6 | file-simple | 10000 | 4.859 | |||
| 4.8.0-SNAPSHOT | dbernstein | daa11f3 | 6 | file-simple | 10000 | 4.841 |
...
| FCREPO Version | Repo | Branch | Commit | modeshape | Environment | # of relations | Test Duration (seconds) | Tester | |
|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | file-simple | 1 | 1000 | 1.408 | ||
| 4.7.1 | fcrepo4 | master | 546f5a5 | file-simple | 2 | 1000 | 1.45 | ||
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | file-simple | 1 | 10,000 | 28.583 | ||
| 4.7.1 | fcrepo4 | master | 546f5a5 | file-simple | 2 | 10,000 | 24.79 | ||
| 4.8.0-SNAPSHOT | fcrepo4 | master | b60d4e | file-simple | 3 | 10,000 | 9.58 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 3 | 10,000 | 12.16 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 4 | 10,000 | 9.71 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 5 | 10,000 | 8.51 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | b60d4e | file-simple | 5 | 10,000 | 8.815 | ||||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | file-simple | 6 | 10,000 | 8.29 | Danny Bernstein | parallel streams enabled. |
| 4.8.0-SNAPSHOT | b60d4e | file-simple | 6 | 10,000 | 8.661 | ||||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 1000 | 1.543 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 10000 | 61.381 | perhaps postgres needs caching configured? | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 1000 | 0.592 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 10000 | 39.671 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402dbernstein | f0a51e | jdbc-postgresql | 3 | 1000 | 4.435 | ||
| 4.8.0-SNAPSHOT | bbranandbernstein | fcrepo-2402 | f0a51e | jdbc-postgresql | 3 | 10000 | 39.486 | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | f453a | jdbc-postgresql | 3 | 1000 | 0.561 | 1 million item cache | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | f453a | jdbc-postgresql | 3 | 10000 | 5.334 | 1 million item cache | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | jdbc-postgresql | 3 | 1000 | 0.633 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | jdbc-postgresql | 3 | 10000 | 5.398 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | mysql-postgresql | 3 | 1000 | 0.820 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | 5138b4 | mysql-postgresql | 3 | 10000 | 7.726 | 1 million item cache + parallelized |
...
Number of relations: 1000
| FCREPO Version | Repo | branch | Commit | modeshape config | Environment | # of relations | Test Duration (seconds) | Tester | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | 1 | 1000 | 6.570 | |||
| 4.7.1 | fcrepo4 | master | 546f5a5 | 2 | 1000 | 2.80 | |||
| 4.8.0-SNAPSHOT | fcrepo4 | master | 2df32 | 1 | 10,000 | 86.000 | |||
| 4.7.1 | fcrepo4 | master | 546f5a5 | 2 | 10,000 | 57.35 | |||
| 4.8.0-SNAPSHOT | fcrepo4 | master | b60d4e | 3 | 10,000 | 30.02 | Unlike the n-member example, results begin streaming right away - so the response begins streaming within 2 seconds. | ||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | 3 | 10,000 | 29.975 | Danny Bernstein | parallel streams enabled. | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | 4 | 10,000 | 24.790 | Danny Bernstein | parallel streams enabled. | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | 5 | 10,000 | 20.992 | Danny Bernstein | parallel streams enabled. | |
| 4.8.0-SNAPSHOT | fcrepo4 | master | b60d4e | 5 | 10,000 | 26.357 | |||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-v4-parallelization | daa11f3 | 6 | 10,000 | 20.337 | Danny Bernstein | parallel streams enabled. | |
| 4.8.0-SNAPSHOT | fcrepo4 | master | b60d4e | 6 | 10,000 | 25.782 | |||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 1000 | 11.414 | ||
| 4.8.0-SNAPSHOT | bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 1 | 10000 | 194 | ||
| 4.8.0-SNAPSHOT | dbernstein | crepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 1000 | 3.961 | parallel streams enabled. | |
| 4.8.0-SNAPSHOT | dbernstein | crepo-2105-v4-parallelization | daa11f3 | jdbc-postgresql | 3 | 10000 | 53.452 | parallel streams enabled. | |
| 4.8.0-SNAPSHOT |
| bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 3 | 1000 | 10.833 | ||
| 4.8.0-SNAPSHOT |
| bbranan | fcrepo-2402 | f0a51e | jdbc-postgresql | 3 | 10000 | 109.530 | |||
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105-large-cache | f453a | jdbc-postgresql | 3 | 10000 | 22.963 | 1 million item cache. | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105- |
| v4- |
| parallelization | 5138b4 | jdbc-postgresql | 3 | 10000 | 11.559 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105- |
| v4- |
| parallelization | 5138b4 | mysql-postgresql | 3 | 1000 | 2.337 | 1 million item cache + parallelized | |
| 4.8.0-SNAPSHOT | dbernstein | fcrepo-2105- |
| v4- |
| parallelization | 5138b4 | mysql-postgresql | 3 | 10000 | 14.998 | 1 million item cache + parallelized |
n-uris.sh
| FCREPO Version | Commit | Environment | # of relations | Test Duration (seconds) | |
|---|---|---|---|---|---|
| 4.8.0-SNAPSHOT | 2df32 | 1 | 1000 | 0.039 | |
| 4.7.1 | 546f5a5 | 2 | 1000 | 0.14 | |
| 4.8.0-SNAPSHOT | 2df32 | 1 | 10,000 | 0.063 | |
| 4.7.1 | 546f5a5 | 2 | 10,000 | 0.17 |
...
| repo | branch | Commit | Environment | Modeshape Config | # of binaries | size in KB | Test Duration (seconds) | Tester |
|---|---|---|---|---|---|---|---|---|
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | file-s3 | 1000 | 1000 | 00:06:02 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | file-simple | 1000 | 1000 | 00:02:16 | |
| https://github.com/bbranan/fcrepo4.git | fcrepo-2402 | f0a51e | 1 | jdbc-postgresql | 1000 | 1000 | 00:02:13 |