The following tests are designed to demonstrate how "performance" of the Fedora repository changes as the size of the repository grows under various, but discrete uses.
Note: "Performance" will be measured by requesting CRUD operations after every x-number of ingest events.
Setup
Along with the following tests, the following items will also be documented:
- Version of Fedora
- Fedora configuration details
- System details (OS, memory, processors, hardware specs or virtualization, JVM, etc)
- Initial state of the repository - likely empty
- Number of client processes/threads (ideally clients are on a separate machine)
Tests
The following measurements should be recorded:
- Number of resources in the repository
- Average response times of the 1000-request blocks
- Individual response times of the performance requests (see below)
- System resource usage on an on-going basis or at the 1000-request boundaries, as appropriate
- memory
- inode count
- disk usage (including swap space)
- garbage collection
- CPU usage
Each of the tests below should be run with different numbers of clients
- First run: 1 client
- Second run: 2 clients
- Third run: 4 clients
- Fourth run: 8 clients
All tests should be scripted using the JMeter framework. The additional advantage is the potential use of those same JMeter scripts to outsource large-scale testing to https://blazemeter.com/.
- For informational purposes, outdated Fedora4/JMeter scripts exist as a starting point: https://github.com/fcrepo4-archive/ff-jmeter-madness
Test Completion
All of the tests below should continue to run until one of the following events take place:
- A pre-defined number of resources have been created
- The collective response time of the performance requests exceeds 1 minute
- Garbage collection is running non-stop
Test 1 - Size of files - large
- Load different "large" files with POST requests at the top of the repository ranging between 10KB and 10GB
- After every 1000 requests, performance requests should be made
Note: Fedora stores files based on their SHA1. In order for this test to be effective, each file should have a different SHA1.
Test 2 - Size of files - small
- Load different "small" files with POST requests at the top of the repository all below the default 4096 Byte threshold
- After every 1000 requests, performance requests should be made
Note: Fedora stores files based on their SHA1. In order for this test to be effective, each file should have a different SHA1.
Test 3 - Number of files
- Load different files with POST requests at the top of the repository ranging between 10KB and 100KB
- After every 1000 requests, performance requests should be made
Test 4 - Number of containers - default
Create containers with POST requests at the top of the repository
After every 1000 requests, performance requests should be made
- Results
Test 5 - Number of containers - RDF bodies
Create containers with POST requests with RDF bodies at the top of the repository
- After every 1000 requests, performance requests should be made
Test 6 - Number of mixed resources - files and containers
Create a mix of resources representative of a typical LDP model including containers and files
- After every 1000 requests, performance requests should be made
Performance requests
- PUT a new resource
- OPTIONS the resource
- GET the resource
- PATCH a property to the resource
- DELETE the resource