Setup
- Version of Fedora
- 4.5.1-SNAPSHOT | Build #fd6f9da9 (2016-02-19) (https://github.com/fcrepo4/fcrepo4/pull/986 before squashing)
- Fedora Configuration
JAVA_OPTS="-Dfcrepo.home=/var/lib/tomcat7/fcrepo4-data -Djava.awt.headless=true -Xmx8g"
See below
- System details
- Fedora: VMWare VM configured with 4x 2.8GHz CPU, 16GB RAM
- JMeter: separate identical VM
Network: 1000Mb/s ethernet
OS: Ubuntu 14.04.3 LTS
JVM: Oracle JDK 1.8.0_66-b17
Servlet container: Tomcat 7.0.52
- 100GB disk, about 70GB free at beginning of test
- Fedora: VMWare VM configured with 4x 2.8GHz CPU, 16GB RAM
- Initial State of the Repository
- empty
- Number of Client Threads
- 1
Test
$HOME/jmeter/bin/jmeter -Dfedora_4_server=lib-fedora1.princeton.edu -Dfedora_4_context=fcrepo/rest -Dfilesize_min=0 -Dfilesize_max=4096 -Dbinary_threads=1 -n -t $HOME/jmeter/fedora.jmx
Results
LevelDB
- Summary:
523,751 in 14,350s = 36.5/s, Avg: 22, Min: 8, Max: 3,738, Err: 1 (0.00%)
- Status: Test ended when LevelDB encountered a "Too many open files" error
- Logs:
- jmeter log: test2-leveldb.log
- jmeter data: test2-leveldb.csv.gz
- jmeter perf: test2-leveldb-perf.log
PostgreSQL
- Additional config:
JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.modeshape.configuration=classpath:/config/jdbc-postgresql/repository.json"
- PostgreSQL 9.3 running on same VM as Fedora
- Summary:
3,959,096 in 293,495s = 13.5/s, Avg: 69, Min: 8, Max: 24243, Err: 1 (0.00%)
- Status: Test ended when Tomcat encountered "java.lang.OutOfMemoryError: GC overhead limit exceeded" error
- Restarting Tomcat restored the repository to a usable state
- Logs:
- jmeter log: test2-postgres.log
- jmeter data: test2-postgres.csv.gz
- jmeter perf: test2-postgres-perf.log
- fedora log: test2-postgres-fedora.log
PostgreSQL with UseG1GC
- Additional config:
JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.modeshape.configuration=classpath:/config/jdbc-postgresql/repository.json -XX:+UseG1GC -XX:+DisableExplicitGC"
- PostgreSQL 9.3 running on same VM as Fedora
- Summary: 4,153,187 in 356,680s = 11.6/s, Avg: 81, Min: 8, Max: 66,544, Err: 1 (0.00%)
- Status: Test ended when the performance CRUD operations timed out
- Tomcat error log contains a WARN message at the same time: "WARN 12:55:39.576 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:aec8:573f1629:fc7cda in state RUN"
- Logs:
- jmeter log: test2-postgres-gc1.log
- jmeter data: test2-postgres-gc1.csv.gz
- fedora log: test2-postgres-gc1-fedora.log
PostgreSQL with Modeshape5 branch
- Version of Fedora:
modeshape5
branch as of 6/5/16, commit 7477d568
- Additional config:
JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.modeshape.configuration=classpath:/config/jdbc-postgresql/repository.json -XX:+UseG1GC -XX:+DisableExplicitGC"
- PostgreSQL 9.5 running on same VM as Fedora
- Note: PostgreSQL upgraded from 9.3 to 9.5 to fix an error only seen with the modeshape5 branch
- Summary:
7,779,352 in 280,285s = 27.8/s, Avg: 31, Min: 1, Max: 3,414, Err: 1 (0.00%)
- Status: Test ended when disk was filled up with PostgreSQL database files.
- Logs:
- jmeter log: test2-postgres-mode5.log
- jmeter data: test2-postgres-mode5.csv.gz
- jmeter perf: test2-postgres-mode5-perf.log
5 Comments
Andrew Woods
Esmé Cowles, Can you share your
JAVA_OPTS
used for the PostgreSQL test?Esmé Cowles
The full JAVA_OPTS are:
-Dfcrepo.home=/var/lib/tomcat7/fcrepo4-data -Djava.awt.headless=true -Xmx8g
-Dfcrepo.modeshape.configuration=classpath:/config/jdbc-postgresql/repository.json
-Dfcrepo.ispn.postgresql.username=******** -Dfcrepo.ispn.postgresql.password=********
Andrew Woods
These have been gaining favor:
-XX:+UseG1GC -XX:+DisableExplicitGC
See: Java HotSpot VM Options recommendations
Esmé Cowles
I was just updating my config with UseG1GC, so I've added both of those options and I'll retry Postgres first, then LevelDB to see what difference they make.
Andrew Woods
It is also worth noting that Aaron Birkland's PR just went into master that should show significant memory benefits: