Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Storage environment:  for the purposes of this test (and for our real migration), we are migrating from one CIFS-mounted remote filesystem to another CIFS-mounted remote filesystem.
  2. Speed: The tool migrates approximately 1700 x objects/hr.  At this rate, it will take approximately 10 x days to migrate the entire repository,
  3. Datastream index:  takes about 67 x minutes to build.
  4. CPU time. Consumes about 15%x%.
  5. Source layout.  Akubra hash storage, using the pattern "#/##/##" for both datastreams and objects.
  6. OCFL storage:  Pairtree.  It will be good when the OCFL storage profile specification is set and incorporated into migration-utils, so that we can define the OCFL layout, similar to how we can specify the Akubra filesystem layout.

...

Fedora 3.8.1.  Migration run on desktop workstation with 8 4 cores, 16 8 GB RAM.  CentOS Linux release 78.72.1908 2004 (Core), Intel(R) CoreXeon(TM) i7-6700 CPU @ 3.40GHzR) Gold 5220 CPU @2.20GHz

Command run:

Code Block
languagebash
titleUW Madison migration-util command line
$ java -jar target/migration-utils-4.4.1-SNAPSHOT-driver.jar --source-type=akubra --datastreams-dir=/fedora3-prod/fedora/datastreams --objects-dir=/fedora3-prod/fedora/objects --target-dir=/fedora-migration-test --layout=pairtree --index-dir=/var/tmp/datastream-index


Number

of objects

Execution

Time

Source

Layout

Dest.

Layout

Migration

tool version

Notes
Datastream index
1 hr 7 minutes

Akubrapairtree
02/02/20 (cd7ece7)233MB100037 min

X MB
1000
Akubrapairtree
02/02/20 (cd7ece7)
1K fedora items produced 33.5K+ files


100,000
59 hours 58 min

Akubrapairtree
02/02/20 (cd7ece7)Stopped and restarted;  time excludes read time for datastream index on startup (29 minutes)


All 390,000
X hours

Akubrapairtree
02/02/20 (cd7ece7)