What are we testing?
The performance of Fedora when performing create, read and delete operations of Fedora objects and datastreams using Fedora's REST API.
See FF+Candidate+Test+Harness for information regarding the test harness, an explanation of the test results and analysing the test results
Test System
Fedora version: 3.6.2 installed on futures6 (see Test Platform for details)
...
- This was gathered using the script gatherSystemInfo.sh
Test Harness
The test was done using Apache JMeter
...
The test script fedoraTest.sh was used to execute the test code with the different test parameters - file size and thread count
The test scripts, results and the graphs generated are available on github at https://github.com/futures/ff-jmeter-testResults/tree/master/2013-02-10-FedoraTests
Test Parameters
Number of Threads: The number of concurrent requests Jmeter will generate, when executing the tests
...
File size mean and standard deviation - Used to produce random binary data, from the stable set of file sizes
...
| Parameter | Values | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Number of threads | 1 | 5 | 10 | 20 | 40 | 80 | 160 | |||||||||||||
| Number of loops | 10 | |||||||||||||||||||
| File size - Mean (bytes) | 1 | 1024 (1k) | 8192 (8k) | 104858 (10k100k) | 1048576 (0.1M) | |||||||||||||||
File size - Standard deviation | 0.1 | 102 | 819 | 10486 | 104858 | |||||||||||||||
The binary data created during the test run, is used as the data for datastreams, in the creation and modification of the datastreams within an object. The size of the binary data is based on the test parameters of mean and standard deviation.
Test Results
- The test was run a total of 35 times (7 values of thread count and 5 values of file size)
- The results are logged to csv files and have been placed in the log directory
- Graphs were generated using R for statistical computing and graphics
A stacked bar chart of the different response codes received, grouped by thread count
A box plot of Thread count vs Time taken to complete a response
A box plot of Thread count vs Time taken to complete a response, for each of the operations
Analysing the test results
The stats visualizations were done using R (http://www.r-project.org/).
...