The following tests

...

Setup

The following items will be documented along with the tests:

• Version of Fedora
• Fedora configuration details
• System details (OS, memory, processors, hardware specs or virtualization, JVM, etc)
• Initial state of the repository - likely empty
• Number of client processes/threads (ideally clients are on a separate machine)

Tests

The following measurements should be recorded:

1. Number of resources in the repository
2. Average response times of the 1000-request blocks
3. Individual response times of the performance requests (see below)
4. System resource usage on an on-going basis or at the 1000-request boundaries, as appropriate
   1. memory
   2. inode count
   3. disk usage (including swap space)
   4. garbage collection
   5. CPU usage

Each of the tests below should be run with different numbers of clients

1. First run: 1 client
2. Second run: 2 clients
3. Third run: 4 clients
4. Fourth run: 8 clients

All tests should be scripted using the JMeter framework. 

• Previous test plans are available in https://github.com/fcrepo4-labs/fcrepo4-jmeter
• For informational purposes, outdated Fedora4/JMeter scripts also exist as a starting point: https://github.com/fcrepo4-archive/ff-jmeter-madness

Test Completion

...

are used for measuring Fedora's limits of scale and stress. The approach of these tests is to continuously ingest variously sized files and containers while measuring response times of performance requests at intervals. The tests continue to run until one of the following "completion events

...

" occurs:

• A

...

• predefined number of resources have been created
• The collective response time of the performance requests exceeds 1 minute
• Garbage collection is running non-stop

Further reading

Test 1 - Size of files - large

1. Load different "large" files with POST requests at the top of the repository ranging between 10KB and 10GB
2. After every 1000 requests, performance requests should be made

Note: Fedora stores files based on their SHA1. In order for this test to be effective, each file should have a different SHA1.

Test 2 - Size of files - small

1. Load different "small" files with POST requests at the top of the repository all below the default 4096 Byte threshold
2. After every 1000 requests, performance requests should be made

Note: Fedora stores files based on their SHA1. In order for this test to be effective, each file should have a different SHA1.

Test 3 - Number of files

1. Load different files with POST requests at the top of the repository ranging between 10KB and 100KB
2. After every 1000 requests, performance requests should be made

Test 4 - Number of containers - default

1. Create containers with POST requests at the top of the repository

2. After every 1000 requests, performance requests should be made

Test 5 - Number of containers - RDF bodies

1. Create containers with POST requests with RDF bodies at the top of the repository

2. After every 1000 requests, performance requests should be made

Test 6 - Number of mixed resources - files and containers

1. Create a mix of resources representative of a typical LDP model including containers and files

2. After every 1000 requests, performance requests should be made

Performance requests

1. PUT a new resource
2. OPTIONS the resource
3. GET the resource
4. PATCH a property to the resource
5. DELETE the resource

Logs

...

• {test-name}-summary.log
• apache-jmeter-5.2.1/bin/log/perf.log
• apache-jmeter-5.2.1/bin/log/jmeter-{test-name}-threads.csv

...

• catalina.out
• local_access_log.txt
• java-gc.log

...

• error.log

• Full (legacy) description of tests: Performance and Scalability Test Plans
• Full (legacy) description of setup/execution of tests: How to Conduct Performance Tests and Share Results

Instructions

These instructions include a "quick start" to get an initial, experimental, testing environment setup. Once the basic setup/execution details have been established, the "real" tests should be performed on a production-like system.

Quick Start

1. Download / Install JMeter (verified with JMeter 2.13 through 5.3)
   1. Unzip/tar the download
   2. Executable: bin/jmeter
2. Download Fedora JMeter test scripts

   1. The important files from the clone are:  "fedora.jmx" and "sample.ttl"

      No Format
      git clone https://github.com/fcrepo4-labs/fcrepo4-jmeter.git

      
      

3. Download / Install Fedora
   1. Deploying Fedora - Complete Guide
4. Run JMeter test(s)

   1. For example:

      No Format
      jmeter -Dfedora_4_server=<default=localhost> -Dfedora_4_context=<default=rest> -Dfilesize_min=0 -Dfilesize_max=4096 -Dbinary_threads=1 -n -t <path/to/fcrepo4-jmeter>/fedora.jmx

      
      

Full Instructions

1. Steps #1 and #2 from "Quick Start" are the same
2. Regarding step #3 (Installing Fedora), it is important to configure the Java HotSpot VM Options, as appropriate
3. Document both the server and client machine specs
   1. Performance Test Result template
   2. Determining Hardware Properties
4. Run JMeter tests (2-5)
   1. Tests #1 and #6 can be skipped

...