| Table of Contents |
|---|
Alpha-1 Test Results
The Alpha-1 benchmark test results are from June:
...
These tests ingested a very small number of objects (10), and files ranging from 1 byte to 100MB.
Alpha-3 Test Parameters
- Number of Objects: In some initial testing on my laptop, I noticed , all nodes were created as children of a single node. This resulted in degraded ingest performance as the repository size grew, particularly after 20K objects.AWoods: I assume this is due to those nodes sharing the same parent node. When ingesting content, do we not sort the nodes into a small hierarchy?
- Datastream Size: In my experience, the larger files in those tests (1MB, 8MB, 100MB) are fairly typical for high-res images and audio/video derivatives, and audio/video masters are much larger.
...
- Thread Test: Run the same test on the same hardware with increasing numbers of client threads
- Fedora 3/4 Comparison: Run the same tests on the same hardware with both Fedora 3 and 4
Ingest Test Matrix (Profile)
Fedora 3/4 Comparison
| Objects | Datastream | Fedora 3 Time (ms) | Fedora 4 | Notes |
|---|---|---|---|---|
| 100 | 50MB | 389,034 | 281,301 | Performance evaluation on the SCC Cluster (Tests #A & #C (AWoods: What do #A and #C refer to?)included here) |
| 100 | 50MB | 881,883 | Performance evaluation on AWS #C | |
| 100 | 50MB | 123285 | 70,728 | Test - Cluster Setup - Yale #A & #C |
| 100 | 50MB | 57,814 | 134,471 | Test - Platform Profile: Single VM at UNC Chapel Hill |
| 100 | 50MB | 231,663 | 131,143 | Platform: Test - Platform Profile: lib-devsandbox1.ucsd.edu, Repository: Test - Repository Profile: Minimal , average of 100 ingest batches |
Thread Test
| Objects | Datastream | 1 Thread | 2 Threads | 3 Threads | 4 Threads |
|---|---|---|---|---|---|
| 100 | 50MB | ||||
| 25,000 | 100KB |
Using Benchtool
The ingest benchmarking tool is available on github: https://github.com/futures/benchtool
...
| Code Block |
|---|
Complete ingest of 1000 files took 23301 ms |
Ingest/Read
...
/Update/Delete Test Results
| Platform/Repository Profiles | Repository | Objects | Datastream SizeFedora 3 Time | Ingest (ms) | Read (ms) | Update (ms) | Delete Fedora 4 Time (ms) | Notes | 1,000 | 50MB|
|---|---|---|---|---|---|---|---|---|---|---|
| 470,466 | 527,349 | Platform: Test - Platform Profile: lib-devsandbox1.ucsd.edu, Repository: Test - Repository Profile: Minimal, average of 10 read batches |
Update Test Matrix (Profile)
fcrepo3/hydra-jetty | 1,000 | 50 MB | 470,466 | ||
| Objects | Datastream | Fedora 3 Time (ms) | Fedora 4 | Notes | |
|---|---|---|---|---|---|
| 1,000 | 50MB | 2,198,425 | 110,787 | ||
| 287,506Platform: Test - Platform Profile: lib-devsandbox1.ucsd.edu, Repository: Test - Repository Profile: Minimal |
Delete Test Matrix (Profile)
| Objects | Datastream | Fedora 3 Time (ms) | Fedora 4 | Notes |
|---|---|---|---|---|
| 1,000 | 50MB | 10,787 | 39,336 | Platform: Test - Platform Profile: lib-devsandbox1.ucsd.edu, Repository: Test - Repository Profile: Minimal |
Combined Performance Tuning Matrix
AWoods: Is there any reason this table is "combined", but the previous tables have been separated?
| fcrepo4 | 1,000 | 50 MB | 527,349 | 1,287,506 | 39,336 | Platform/Repository Profiles | Repository | Objects | Datastream Size | Ingest (ms) | Read (ms) | Update (ms) | Delete (ms)||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| lib-devsandbox1.ucsd.edu | fcrepo3/hydra-jetty | 1,000 | 10 MB | 485,700 | 107,649 | 438,001 | 52,611 | |||||||
| lib-devsandbox1.ucsd.edu | fcrepo4 (2707674c86e5a82df82f937fb87c6a1a259c0bd3) | 1,000 | 10 MB | 334,177 | 144,367 | 324,908 | 45,334 | |||||||
| lib-devsandbox1.ucsd.edu | barmintor/fcrepo4 (a0a6f1af258f1247c824e30a186f38c2071395a3) (with custom ispn config) | 1,000 | 10 MB | 304,447 | 144,682 | 281,381 | 37,805 | |||||||
| lib-devsandbox1.ucsd.edu + NAS + file | fcrepo4 (ceae23e7a18b30a9aa283e5a66da290a551fe8ab) | 1,000 | 10 MB | 331,110 | 155,743 | 354,122 | 45,119 | |||||||
| lib-devsandbox1.ucsd.edu + NAS + leveldb | fcrepo4 (ceae23e7a18b30a9aa283e5a66da290a551fe8ab) | 1,000 | 10 MB | 344,792 | 167,567 | 366,714 | 45,689 | |||||||
| lib-devsandbox1.ucsd.edu + local disk + file | fcrepo4 (ceae23e7a18b30a9aa283e5a66da290a551fe8ab) | 1,000 | 10 MB | 235,597 | 169,930 | 253,178 | 45,492 | |||||||
| lib-devsandbox1.ucsd.edu + local disk + leveldb | fcrepo4 (ceae23e7a18b30a9aa283e5a66da290a551fe8ab) | 1,000 | 10 MB | 228,773 | 161,836 | 246,275 | 42,535 |