This document details both the categories of tests that help in understanding Fedora's performance and scalability characteristics, as well as the tools used to perform those tests. This documentation is an aggregation of various testing initiatives from within the Fedora community over the last several years. Although the tooling has been updated to work with the most recent version of Fedora (6.0), it is important that the results produced through the current tests be compared to historical results in order to better understand the changes (improvements or degradations) in Fedora's performance characteristics over time.
Note: This testing approach was initially structured by the Product Technology Subgroup's: Fedora 6 Test Plan
What is Fedora's degree of compliance with the Fedora API specification?
What is Fedora's degree of compliance with the OCFL specification? Tests related to this category should be run against both the OCFL produced by Fedora, as well as the OCFL produced by the Fedora 3 to 6 migration tooling.
What are Fedora's ingest performance characteristics? Specifically, how much content can reasonably be stored in Fedora (number of objects, number of petabytes)? How long does it take to ingest a large corpus of data into Fedora?
What are Fedora's read performance characteristics? Specifically, how does the response time of various read requests change as the size of the repository grows?
How do Fedora's ingest and read performance characteristics change with auto-versioning on or disabled?
What are Fedora's query response time characteristics for common and complex queries as the size of the repository grows?
What are the migration times from Fedora 3 to an OCFL representation for a variety of corpus sizes (number of objects, number of petabytes)? What are the rebuild times for Fedora 6 layered over the OCFL representation?
What is the speed of the rebuild with different data profiles, database engines, and instance types?
How is rebuild performance affected by instance size (cpu count), database engine, database instance size?
Current test results can be found on the Fedora 6 Test Results page.
One of the benefits of running these tests is to measure the change in performance characteristics in relation to previous versions of Fedora. Ideally, one of the products of this testing initiative will be to have a clear depiction, across the test categories, of performance result summaries from the past and present.
Although a consolidated summary of previous tests is not available in a single location, the following references point to the locations where previous test results can be found: