| Excerpt |
|---|
Fedora 4 can handle at least 10 million objects, but performance degrades if there are too many children and/or grandchildren of a single node. |
| Table of Contents |
|---|
Repository storage
Objects under a single node
Under a single node, batches of 500 objects were created, each with a single 10KB datastream. After each batch, the entire directory contents were listed using the REST API ("whole"). The test was then repeated using the limit parameter to list the first 1,000 objects ("paged"). Listing the objects scales roughly linearly in both cases, but much more steeply for listing all the children. Using paging, acceptance is acceptable up to 30,000 children with no sharp increase observed.
...
16.7 million files were generated on a federated filesystem using a three-level hierarchy (256 top-level nodes, 256 second-level nodes in each, 256 third-level nodes in each, and one 10KB datastream in each), taking 26.78 hours. After each batch, three REST API operations were timed: listing the top level of the repository ("toplist"), listing a second-level node ("dirlist"), and retrieving a file ("fileget"). Performance retrieving files and listing the second-level nodes did not degrade with larger numbers of objects. However, listing the top-level of the repository degraded roughly linearly as more objects were added, and became increasing erratic.
...