...
10 million files were ingested into a test repository running Fedora 4.0-beta1 (lib-devsandbox1.ucsd.edu) using a four-level hierarchy (39 top-level nodes, 64 second- through fourth-level nodes, and one 10KB datastream in each bottom-level node), taking 54 days (averging 186K objects/day). Ingest was done using a Java program, with a randomized datastream data.
After each batch (256K objects), three REST API operations were timed: listing the top level of the repository ("toplist"), listing a third-level node ("dirlist"), and retrieving a file ("fileget"). Performance retrieving files did not degrade with larger numbers of objects. However, listing the top-level of the repository degraded roughly linearly as more objects were added, and listing a third-level node increased more rapidly, with increasing variability as more objects were created.
...