...
Category | User Operation | Repository Operation | Test Metric | Test | Priority | Notes |
|---|---|---|---|---|---|---|
| Authoring | Use concurrency test (see below) | |||||
| Authoring | Authoring with Workflow Not planned for initial test | |||||
| Simple Ingest | Small Files | Rate | 1 | Synthetic data is acceptable for this test | ||
| Simple Ingest | Medium Files | Rate | 1 | Synthetic data is acceptable for this test | ||
| Simple Ingest | Large Files | Rate | 1 | Synthetic data is acceptable for this test | ||
| Simple Ingest | Mixed Files | Rate | 1 | Synthetic data is acceptable for this test | ||
| Simple Ingest | Media Files | |||||
| Simple Ingest | Large File Count | Rate - Normalized Count | Ingest files to a substantial number to explore maximum file count. Normalized to ignore size of a given file. | |||
| Bulk Ingest | Small Files | |||||
| Bulk Ingest | Medium Files | |||||
| Bulk Ingest | Large Files | |||||
| Bulk Ingest | Large File Count | Rate - Normalized Count | Ingest files to a substantial number to explore maximum file count | |||
| Simple Access | Small Files | Random Access | 1 | Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test. | ||
| Simple Access | Medium Files | Random Access | 1 | Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test. | ||
| Simple Access | Large Files | Random Access | 1 | Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test. | ||
| Simple Access | Mixed Files | Random Access | 1 | Site should contain a set of files of all three sizes. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test. | ||
| Simple Access | Large File Count | Random Access | 1 | Access a large number of files until time is exhausted. | ||
| Bulk Ingest | Small Files | |||||
| Bulk Ingest | Medium Files | |||||
| Bulk Ingest | Large Files | |||||
| Bulk Ingest | Mixed Files | |||||
| Conditioned Access | None planned for initial testing | |||||
| Mediated Access | None planned for initial testing | |||||
| Concurrent Test #1 | Simple Access Mixed Files Simple Ingest Mixed Files | Rate Random Access | 1 | Number of Load Injectors TBD Number of Load Injectors TBD | ||
| Concurrent Test #1 | Authoring Simulation | Rate Directed Access | Random ingests (writes), Random delay read of same file. Count errors. | |||
Testing Tools and Configuration
The suggested testing tool is The Grinder. It is a load testing framework written in Java, that uses Java, Jython and/or Clojure for writing the tests. A more complete discussion of our testing infrastructure may be found on this page.
Testing Considerations
These scenarios expand on the previous single load injector tests to use multiple read, write, and read-write tests via the REST api.
...
a steady state is achieved
a declining state is achieved
Fedora 4 no longer responds
Multiple ReadLoad Injectors
Stimulators
1 stimulator to Load Injector to provide a baseline similar to the previous testing regimen
3 stimulators3 Load Injectors
6 stimulators Load Injectors (since this is where Fedora 3 starts to exhibit limits)
12 stimulators Load Injectors (since this is where Fedora 3 always exhibits limits)24 stimulators
24 Load Injectors
Payloads
1K file
1M file
50M file (Avg Video)
2.7G file (DVD)
...
Step up rates X2 until flat line
Then proceed to declining response and failure or non- response
Multiple Write
Stimulators
1 stimulator to provide a baseline similar to the previous testing regimen
3 stimulators
6 stimulators (since this is where Fedora 3 starts to exhibit limits)
12 stimulators (since this is where Fedora 3 always exhibits limits)
24 stimulators
Payloads
1K file
1M file
50M file (Avg Video)
2.7G file (DVD)
Rates
Step up rates X2 until flat line
Then procede to declining response and failure or non-response
Read-Write
Stimulators
1 stimulator to provide a baseline similar to the previous testing regimen
3 stimulators
6 stimulators (since this is where Fedora 3 starts to exhibit limits)
12 stimulators (since this is where Fedora 3 always exhibits limits)
24 stimulators
Payloads
This needs to be matrixed. The payloads should be mixed but not randomly to make the tests repeatable.
1K file
1M file
50M file (Avg Video)
2.7G file (DVD)
Rates
Step up rates X2 until flat line
Then proceed to declining performance and failure or non-response
Fedora Configurations
- Not Clustered
- Clustered
- Replicated
- Not Replicated
...