...
The tests were run on an EC2 m3.medium instance. Tests were run with the pre-configured Infinispan configurations available in the fcrepo4/fcrepo-configs repository (e.g., file, leveldb-default, leveldb, and ram). These configurations were passed in as system properties when Fedora (via the jetty-maven-plugin) was started. Once Fedora was up, benchtool was used to ingest up to 30,000 objects and the resulting durations.log file was processed through gnuplot to generate a graph. A Bash script was written to automate this process. In the tests, ingest times across the various Infinispan configurations did not slow down significantly as more objects were ingested. Configurations, graphs, and benchtool log output are available below.
Conclusion: Based on the tests The initial tests, documented below, the Infinispan backends do not seem to have did not indicate a problem with a lack of hierarchy within the tested ranges. The hypothesis was that we'd see a significant slowdown around the ten thousand object mark, but the tests below seem to scale to the 30k object mark without significant slowdowns, though there is a slight uptick towards the end of the 30k 2MB object tests. The tests below were run with benchtool's the default object size (1 KB) and, a second time, with 2 MB objects. With the uptick at the end of the larger 30k object tests, more testing is probably warranted.
Tests with 500k, 100k, and 60k objects were attempted, but were not able to be completed. The result of these attempts was a ticket to investigate an ExecutionError (seemingly related to transactions).
Update: Since there was a slight uptick at the end of the 30k 2 MB test, and there were problems with the larger ingests when using benchtool, the bash script mentioned above was modified to use curl instead of benchtool. Ingests times for the curl ingests were output in the same way that benchtool does, and then run through gnuplot to generate a graph. The first test run was for 100k records. Only the leveldb-default configuration was used. Unlike the benchtool test, this test did not use multiple threads.
This test does show an ongoing gradual increase in the amount of time ingests take. The graph for this test is at the bottom of the page.
A new test to measure the ingest times with 1 million objects is underway. It's graph will be posted to this page when it's completed.
Configuration | Results Graph | Logging Output |
---|
leveldb-default (30k 1KB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/leveldb-default-durations-30000-1.png?version=1&modificationDate=1397283018844&api=v2) | Running new test [config: config/infinispan/leveldb-default/infinispan.xml] [Jetty pid: 30298] 18:30:11 INFO Found Fedora 4 at http://localhost:8080 18:30:11 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 1.0 KB using 15 thread(s) 18:30:15 INFO The Fedora cluster has 0 node(s) before the benchmark 18:30:15 INFO preparing 30000 objects 18:32:30 INFO creating 30000 objects took 125639 ms 18:33:27 INFO scheduling 30000 actions 19:08:14 INFO purging 30000 objects and datastreams 19:10:38 INFO Completed 30000 INGEST action(s) executed in 31292451 ms 19:10:38 INFO The Fedora cluster has 0 node(s) after the benchmark 19:10:38 INFO Throughput was 0.01 MB/sec 19:10:38 INFO Throughput per thread was 0 MB/sec 19:10:38 INFO Condensed results: 19:10:38 INFO 30000 1024 15 INGEST 31292451 9.3622814E-4 no-tx 19:10:38 INFO All operations completed in 2426697 ms |
leveldb (30k 1KB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/leveldb-durations-30000-1.png?version=1&modificationDate=1397283048522&api=v2) | Running new test [config: config/infinispan/leveldb/infinispan.xml] [Jetty pid: 3870] 19:12:40 INFO Found Fedora 4 at http://localhost:8080 19:12:40 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 1.0 KB using 15 thread(s) 19:12:44 INFO The Fedora cluster has 0 node(s) before the benchmark 19:12:44 INFO preparing 30000 objects 19:15:01 INFO creating 30000 objects took 126888 ms 19:16:04 INFO scheduling 30000 actions 19:51:06 INFO purging 30000 objects and datastreams 19:53:28 INFO Completed 30000 INGEST action(s) executed in 31517411 ms 19:53:29 INFO The Fedora cluster has 0 node(s) after the benchmark 19:53:29 INFO Throughput was 0.01 MB/sec 19:53:29 INFO Throughput per thread was 0 MB/sec 19:53:29 INFO Condensed results: 19:53:29 INFO 30000 1024 15 INGEST 31517411 9.295457E-4 no-tx 19:53:29 INFO All operations completed in 2448012 ms |
file (30k 1KB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/file-durations-30000-1.png?version=1&modificationDate=1397283086124&api=v2) | Running new test [config: config/infinispan/file/infinispan.xml] [Jetty pid: 7558] 20:02:31 INFO Found Fedora 4 at http://localhost:8080 20:02:31 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 1.0 KB using 15 thread(s) 20:02:34 INFO The Fedora cluster has 0 node(s) before the benchmark 20:02:34 INFO preparing 30000 objects 20:04:47 INFO creating 30000 objects took 124113 ms 20:08:28 INFO scheduling 30000 actions 20:45:14 INFO purging 30000 objects and datastreams 20:50:51 INFO Completed 30000 INGEST action(s) executed in 33069648 ms 20:50:52 INFO The Fedora cluster has 0 node(s) after the benchmark 20:50:52 INFO Throughput was 0.01 MB/sec 20:50:52 INFO Throughput per thread was 0 MB/sec 20:50:52 INFO Condensed results: 20:50:52 INFO 30000 1024 15 INGEST 33069648 8.859143E-4 no-tx 20:50:52 INFO All operations completed in 2900327 ms |
ram (30k 1KB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/ram-durations-30000-1.png?version=1&modificationDate=1397283106155&api=v2) | Running new test [config: config/infinispan/ram/infinispan.xml] [Jetty pid: 11324] 20:52:45 INFO Found Fedora 4 at http://localhost:8080 20:52:45 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 1.0 KB using 15 thread(s) 20:52:48 INFO The Fedora cluster has 0 node(s) before the benchmark 20:52:48 INFO preparing 30000 objects 20:55:00 INFO creating 30000 objects took 123502 ms 20:55:45 INFO scheduling 30000 actions 21:28:38 INFO purging 30000 objects and datastreams 21:30:23 INFO Completed 30000 INGEST action(s) executed in 29588918 ms 21:30:23 INFO The Fedora cluster has 0 node(s) after the benchmark 21:30:23 INFO Throughput was 0.01 MB/sec 21:30:23 INFO Throughput per thread was 0 MB/sec 21:30:23 INFO Condensed results: 21:30:23 INFO 30000 1024 15 INGEST 29588918 9.9013E-4 no-tx 21:30:23 INFO All operations completed in 2257950 ms |
Configuration | Results Graph | Logging Output |
---|
leveldb-default (30k 2MB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/leveldb-default-durations-30000.png?version=5&modificationDate=1397283148917&api=v2) | Running new test [config: config/infinispan/leveldb-default/infinispan.xml] [Jetty pid: 1370] 14:10:27 INFO Found Fedora 4 at http://localhost:8080 14:10:27 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 2.0 MB using 15 thread(s) 14:10:29 INFO The Fedora cluster has 0 node(s) before the benchmark 14:10:29 INFO preparing 30000 objects 14:12:40 INFO creating 30000 objects took 121853 ms 14:13:34 INFO scheduling 30000 actions 15:48:36 INFO purging 30000 objects and datastreams 15:51:10 INFO Completed 30000 INGEST action(s) executed in 85507949 ms 15:51:10 INFO The Fedora cluster has 0 node(s) after the benchmark 15:51:10 INFO Throughput was 10.53 MB/sec 15:51:10 INFO Throughput per thread was 0.7 MB/sec 15:51:10 INFO Condensed results: 15:51:10 INFO 30000 2097152 15 INGEST 85507949 0.7016891 no-tx 15:51:10 INFO All operations completed in 6042901 ms |
leveldb (30k 2MB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/leveldb-durations-30000.png?version=5&modificationDate=1397283273737&api=v2) | Running new test [config: config/infinispan/leveldb/infinispan.xml] [Jetty pid: 7694] 18:06:30 INFO Found Fedora 4 at http://localhost:8080 18:06:30 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 2.0 MB using 15 thread(s) 18:06:32 INFO The Fedora cluster has 0 node(s) before the benchmark 18:06:32 INFO preparing 30000 objects 18:08:41 INFO creating 30000 objects took 120016 ms 18:09:34 INFO scheduling 30000 actions 19:45:21 INFO purging 30000 objects and datastreams 19:47:52 INFO Completed 30000 INGEST action(s) executed in 86194079 ms 19:47:52 INFO The Fedora cluster has 0 node(s) after the benchmark 19:47:52 INFO Throughput was 10.44 MB/sec 19:47:52 INFO Throughput per thread was 0.7 MB/sec 19:47:52 INFO Condensed results: 19:47:52 INFO 30000 2097152 15 INGEST 86194079 0.6961035 no-tx 19:47:52 INFO All operations completed in 6081741 ms |
file (30k 2MB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/file-durations-30000.png?version=3&modificationDate=1397273710191&api=v2) | Running new test [config: config/infinispan/file/infinispan.xml] [Jetty pid: 14070] 21:45:20 INFO Found Fedora 4 at http://localhost:8080 21:45:20 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 2.0 MB using 15 thread(s) 21:45:22 INFO The Fedora cluster has 0 node(s) before the benchmark 21:45:22 INFO preparing 30000 objects 21:47:36 INFO creating 30000 objects took 124645 ms 21:51:06 INFO scheduling 30000 actions 23:26:46 INFO purging 30000 objects and datastreams 23:31:50 INFO Completed 30000 INGEST action(s) executed in 86083830 ms 23:31:51 INFO The Fedora cluster has 0 node(s) after the benchmark 23:31:51 INFO Throughput was 10.45 MB/sec 23:31:51 INFO Throughput per thread was 0.7 MB/sec 23:31:51 INFO Condensed results: 23:31:51 INFO 30000 2097152 15 INGEST 86083830 0.696995 no-tx 23:31:51 INFO All operations completed in 6390673 ms |
ram (30k 2MB objects / 15 threads) - tested using benchtool | ![](/download/attachments/54296836/ram-durations-30000.png?version=4&modificationDate=1397345371018&api=v2) | Running new test [config: config/infinispan/ram/infinispan.xml] [Jetty pid: 1573] 17:46:22 INFO Found Fedora 4 at http://localhost:8080 17:46:22 INFO Running 30000 INGEST action(s) against FCREPO4 with a binary size of 2.0 MB using 15 thread(s) 17:46:24 INFO The Fedora cluster has 0 node(s) before the benchmark 17:46:24 INFO preparing 30000 objects 17:48:35 INFO creating 30000 objects took 122076 ms 17:49:25 INFO scheduling 30000 actions 19:18:49 INFO purging 30000 objects and datastreams 19:21:02 INFO Completed 30000 INGEST action(s) executed in 80453741 ms 19:21:02 INFO The Fedora cluster has 0 node(s) after the benchmark 19:21:02 INFO Throughput was 11.19 MB/sec 19:21:02 INFO Throughput per thread was 0.75 MB/sec 19:21:02 INFO Condensed results: 19:21:02 INFO 30000 2097152 15 INGEST 80453741 0.74577016 no-tx 19:21:02 INFO All operations completed in 5680046 ms |
...
Configuration | Results Graph |
---|
leveldb-default (100k 1MB objects / 1 thread) - tested using curl | Image Added
|