CRUD Concurrent Integration Test
Running CRUD integration tests (FedoraCrudConcurrentIT.java in fcrepo-http-api) for simple Glossary and small content files CREATE, UPDATE, DELETE and RETRIEVE with 2, 4, 8, 16, 32 concurrent threads.
The following test threads are executed exactly once. In other words, in the "2 threads" test, each of the two threads performs exactly one operation. Likewise with the "32 threads" test.
The test was performed on 2014-05-20 with the master branch at this commit: https://github.com/fcrepo4/fcrepo4/commit/8af7da1cce5c692d092b1228e8f8c8c1dbd0d06d
It can be seen that the tests below were run with threads numbering from 2 to 32. The resultant graphs show unexpected behavior at the beginning and the ending of most of these tests; hence the running of the tests twice: with both increasing and decreasing numbers of threads.
In any case, the strange behavior in the initial and final test runs indicates a probably issue with the testing framework.
Average CRUD response time (ms) for simple containers
Test result started with threads [2, 4, 8, 16, 32]:
Threads | Create | Update | Retrieve | Delete |
---|---|---|---|---|
2 | 2421 | 483 | 58 | 33 |
4 | 34 | 32 | 20 | 21 |
8 | 44 | 31 | 17 | 17 |
16 | 27 | 31 | 15 | 18 |
32 | 28 | 33 | 12 | 130 |
Test result started with threads [32, 16, 8, 4, 2]:
Threads | Create | Update | Retrieve | Delete |
---|---|---|---|---|
32 | 136 | 83 | 17 | 18 |
16 | 38 | 27 | 12 | 15 |
8 | 18 | 23 | 14 | 14 |
4 | 18 | 23 | 11 | 13 |
2 | 22 | 177 | 10 | 143 |
Average CRUD response time (ms) for content files
Test result started with threads [2, 4, 8, 16, 32]:
Threads | Ingest | Update | Retrieve | Delete |
---|---|---|---|---|
2 | 248 | 74 | 12 | 29 |
4 | 82 | 53 | 12 | 24 |
8 | 57 | 52 | 8 | 21 |
16 | 61 | 39 | 8 | 29 |
32 | 71 | 699 | 35 | 194 |
Test result started with threads [32, 16, 8, 4, 2]:
Threads | Create | Update | Retrieve | Delete |
---|---|---|---|---|
32 | 81 | 56 | 8 | 23 |
16 | 68 | 56 | 8 | 17 |
8 | 55 | 30 | 5 | 21 |
4 | 48 | 34 | 6 | 16 |
2 | 110 | 838 | 5 | 297 |