Versions Compared

Old Version 33

changes.mady.by.user Daniel Davis

Saved on

compared with

New Version 34

changes.mady.by.user Daniel Davis

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Download of large amount of content as single user operation.  This may require any number of repository operations to accomplish. Whether content and metadata stays fixed from the beginning to the end of the operation is to be defined. This is needs consideration a whole intellectual entity, graph or DIP is considered the unit.  Also we need to consider what this means for continuous access operations.

Tests

Baseline and Simple Concurrency Testing Matrix

The tests in the table will start with a single load injector and worker to use as a baseline.  Then each of the tests are executed to test concurrency with increasing numbers of load injectors and workers until performance declines and/or error rates become large.

Category

User OperationRepository OperationTest MetricTestPriorityNotes
Simple IngestSmall Files Rate 1Increase Load Injectors until max rate is found. Synthetic data is acceptable for this test
Simple IngestMedium Files Rate 1Increase Load Injectors until max rate is found. Synthetic data is acceptable for this test
Simple IngestLarge Files Rate 1Increase Load Injectors until max rate is found. Synthetic data is acceptable for this test
Simple IngestMedia Files     
Simple IngestLarge File Count 

Rate - Normalized

Count

 1Ingest files to a substantial number to explore maximum file count. Normalized to ignore size of a given file.
Bulk IngestSmall Files     
Bulk IngestMedium Files     
Bulk IngestLarge Files     
Bulk IngestLarge File Count 

Rate - Normalized

Count

  Ingest files to a substantial number to explore maximum file count
Simple AccessSmall Files Random Access 1Increase Load Injectors until max rate is found. Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test.
Simple AccessMedium Files Random Access 1Increase Load Injectors until max rate is found. Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test.
Simple AccessLarge Files Random Access 1Increase Load Injectors until max rate is found. Site should contain a set of files of uniform size. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test.
Simple AccessMixed Files Random Access 1Increase Load Injectors until max rate is found. Site should contain a set of files of all three sizes. Tests can vary download mix by URL. It is essential that caching be avoided. Synthetic data is acceptable for this test.
Simple AccessLarge File Count Random Access 1Increase Load Injectors until max rate is found. Access a large number of files until time is exhausted.
Bulk IngestSmall Files     
Bulk IngestMedium Files     
Bulk IngestLarge Files     
Bulk IngestMixed Files     

 

Concurrency Tests

The tests in the table combine two or more different tests from the table above. It starts with concurrent operation of at least two workers, one for each simple test.  Then each of the tests are executed with increasing numbers of load injectors and workers until performance declines and/or error rates become large.Concurrency Tests

CategoryUser OperationRepository OperationTest MetricTestPriorityNotes
Concurrent Test #1

Simple Access Mixed Files

Simple Ingest Mixed Files

 

Rate

Random Access

 1

Number of Load Injectors TBD

Number of Load Injectors TBD

       
Concurrent Test #NAuthoring Simulation 

Rate

Directed Access

   

...

These scenarios expand on the previous single load injector tests to use multiple read, write, and read-write tests via the REST api.

If the group think this is useful then I can break it into a matrix.

Note: all tests need to be taken until:

  • a steady state is achieved

  • a  declining state is achieved

  • Fedora 4 no longer respondsexhibits high error rates or stops responding

Load Injectors

  • 1 Load Injector to provide a baseline similar to the previous testing regimen

  • 3 Load Injectors

  • 6 Load Injectors (since this is where Fedora 3 starts to exhibit limits)

  • 12 Load Injectors (since this is where Fedora 3 always exhibits limits)

  • 24 Load Injectors

...

  • Step up rates X2 until flat line

  • Then proceed to declining response and failure , high error rates or non-response

Fedora Configurations

  • Not Clustered
  • Clustered
  • Not ReplicatedNot
  • Replicated

 

Resources