We are currently supporting migration to Fedora 6 from Fedora 3, 4.7.5 and 5.1.1. This document describes the migration process for each of these pathways.
This migration path relies on the migration-utils tool.
The migration-utils tool can migrate either exported FOXML (in archive or migration format), or directly migrate the Fedora 3 data stored on-disk. We recommend the direct Fedora 3 filesystem migration.
The migration-utils tool will not make any changes to your Fedora 3 repository – it only reads data from Fedora 3.
You will need to make sure that you have sufficient storage space available in the target Fedora 6 (OCFL) directory, as the migration will effectively create a copy of your Fedora 3 repository in the Fedora 6 format which conforms to the Oxford Common File Layout (OCFL) specification.
Once your Fedora 3 repository has been migrated to Fedora 6, you may start up an instance of Fedora 6 on top of the newly created OCFL directory tree. As Fedora 6 starts up it will automatically rebuild internal indices by scanning the OCFL tree. This index initialization can take a few seconds or several hours depending on the size of your repository.
Note that the Fedora 3 migration utility will not migrate the following Fedora 3 features:
grep
utility, etc.Download the latest version of the migration-utils utility
Start up Fedora on top of your newly created Fedora 6 OCFL-compliant repository using the -Dfcrepo.home configuration property.
The migration-utils tool is a java command-line tool. Make sure you run the utility with Java 11.
screen
or tmux
session that you can detach from. Try not to shut down or reboot the host while the tool is running.--resume
flag (keeping all the other parameters the same).Redirect the output of the utility to a log file that you can analyze at your leisure during and after the migration. Example:
java -jar migration-utils-6.0.0-driver.jar \ --source-type=legacy \ --target-dir=my-fcrepo-6-home \ --objects-dir=my-fcrepo-3/objects \ --datastreams-dir=my-fcrepo-3/datastreams > log.txt 2>&1 |
Errors: the migration tool may encounter problems copying some objects or datastreams. The tool by default will halt at the first error; if you wish to migrate as much as possible then go back and address errors, run the tool with the --continue-on-error
flag. Objects or datastreams with errors will not be written to the OCFL repository; they will be skipped, and the next object in the list will be processed.
Objects that could not be migrated will provoke a stack dump in the log, marked with the string ERROR. They can be extracted from the log at a later date and fixed, then migrated individually.
Example:
$ grep ERROR log.txt ERROR 01:09:09.801 (Migrator) MIGRATION_FAILURE: pid="test:BadPID1", message="Unable to resolve internal ID "test:BadPID1+MYDS+MYDS.2"!" ERROR 01:29:54.878 (Migrator) MIGRATION_FAILURE: pid="test:BadPID23", message="Unable to resolve internal ID "test:BadPID23+MYDS+MYDS.0"!" ERROR 02:11:50.644 (Migrator) MIGRATION_FAILURE: pid="test:BadPID617", message="Unable to resolve internal ID "test:BadPID617+MYDS+MYDS.1"!" ... |
As many files are created per object and per datastream version, make sure to allocate enough inodes on your system to allow for all the files (note: this should only be necessary for
extremely large repositories, more than 4 million objects, for example).Select 1000 pids or so from your Fedora 3 repository, and migrate just those objects to an OCFL repository. Start up Fedora 6 on the sample OCFL tree, and examine. You may also look at the contents of the OCFL repository on the filesystem – the structure and contents are easily visible to the naked eye. Use the sample run to determine the disk space required, and time needed to migrate your whole repository.
Full migration:
--pid-file
parameter.--continue-on-error
flag.Once you are satisfied with the migration, start up Fedora 6 on top of your new OCFL repository.
Usage: migration-utils [-chrVx] [--debug] -a=<targetDir> [-d=<f3DatastreamsDir>] [-e=<f3ExportedDir>] [-f=<f3hostname>] \ [-i=<indexDir>] [-l=<objectLimit>] [-m=<migrationType>] [-o=<f3ObjectsDir>] [-p=<pidFile>] -t=<f3SourceType> [-u=<user>] [-U=<userUri>] -h, --help Show this help message and exit. -V, --version Print version information and exit. -t, --source-type=<f3SourceType> Fedora 3 source type. Choices: akubra | legacy | exported -d, --datastreams-dir=<f3DatastreamsDir> Directory containing Fedora 3 datastreams (used with --source-type 'akubra' or 'legacy') -o, --objects-dir=<f3ObjectsDir> Directory containing Fedora 3 objects (used with --source-type 'akubra' or 'legacy') -e, --exported-dir=<f3ExportedDir> Directory containing Fedora 3 export (used with --source-type 'exported') -a, --target-dir=<targetDir> Directory where the migrated objects will be written -i, --working-dir=<workingDir> Directory where supporting state will be written (cached index of datastreams, ...) -I, --delete-inactive Migrate objects and datastreams in the Inactive state as deleted. Default: false. -m, --migration-type=<migrationType> Type of OCFL objects to migrate to. Choices: FEDORA_OCFL | PLAIN_OCFL Default: FEDORA_OCFL -l, --limit=<objectLimit> Limit number of objects to be processed. Default: no limit -r, --resume Resume from last successfully migrated Fedora 3 object Default: false -c, --continue-on-error Continue to next PID if an error occurs (instead of exiting). Disabled by default. Default: false -p, --pid-file=<pidFile> PID file listing which Fedora 3 objects to migrate -x, --extensions Add file extensions to migrated datastreams based on mimetype recorded in FOXML Default: false -f, --f3hostname=<f3hostname> Hostname of Fedora 3, used for replacing placeholder in 'E' and 'R' datastream URLs Default: fedora.info -u, --username=<user> The username to associate with all of the migrated resources. Default: fedoraAdmin -U, --user-uri=<userUri> The username URI to associate with all of the migrated resources. Default: info:fedora/fedoraAdmin --debug Enables debug logging |
java -jar migration-utils-<latest-version>-driver.jar \ --source-type=legacy \ --limit=100 \ --target-dir=my-fcrepo-6-home \ --working-dir=<tmp working dir> \ --objects-dir=<path to objects dir> \ --datastreams-dir=<path to datastreams dir> |
java -Dfcrepo.home=my-fcrepo-6-home -jar fcrepo-webapp-<latest fedora 6 version>-jetty-console.jar --headless |
The Fedora 4 → 6 migration path assumes your source repository is version 4.7.5. If you are running an earlier version of Fedora 4.x please see the Upgrade Notes for the steps to upgrade to version 4.7.5. |
Migrating from Fedora 4 → 6 is slightly more complicated than the 3→ 6 path. In a nutshell , you will need to do the following:
Make sure that your Fedora 4.7.5 instance is running. Also be sure that you are using v0.3.0 of the import export tool: ie fcrepo-import-export-0.3.0.jar! Then run the following command (swapping in appropriate local values):
java -jar fcrepo-import-export-0.3.0.jar -b \ -d my-4.7.5-export \ -u fedoraAdmin:fedoraAdmin \ -m export \ -r http://localhost:8080/rest \ --binaries \ --versions |
java -jar fcrepo-upgrade-utils-<latest version>.jar \ -i my-4.7.5-export \ -o my-5.1.1-export \ -s 4.7.5 \ -t 5+ |
# create your destination directory for the upgrade mkdir -p my-fcrepo-6-home java -jar fcrepo-upgrade-utils-<latest-version>.jar \ -i my-5.1.1-export \ -o my-fcrepo-6-home \ -s 5+ \ -t 6+ \ -u http://localhost:8080/rest |
java -Dfcrepo.home=my-fcrepo-6-home -jar fcrepo-webapp--<latest fedora 6 version>-jetty-console.jar --headless |
For migrating from Fedora 5.1.1 you will follow a similar process to the previous section, however note that you will use a different version of the import export tool to export your F5 repository and you will perform only one upgrade. In other words, here are the steps:
Below you will find a sample recipes for this migration path.
Make sure that your Fedora 5 instance is running. Also be sure that you are using v1.0.0 of the import export tool. Then run the following command:
java -jar fcrepo-import-export-1.0.0.jar -b \ -d my-5.1.1-export \ -u fedoraAdmin:fedoraAdmin \ -m export \ -r http://localhost:8080/rest \ --binaries \ --versions |
java -jar fcrepo-upgrade-utils-<latest-version>.jar \ -i my-5.1.1-export \ -o my-fcrepo-6-home \ -s 5+ \ -t 6+ \ -u http://localhost:8080/rest |
java -Dfcrepo.home=my-fcrepo-6-home -jar fcrepo-webapp-<latest fedora 6 version>-jetty-console.jar --headless |
The very first time Fedora starts it initializes the database and indexes any resources on disk. On subsequent starts these steps are not executed. If you started Fedora once and either your migrated content was not in place or there was some other configuration problem that needed addressed, then Fedora initialized to an empty state and it will not see your content until it has been instructed to reindex all of the resources in the repository.
You can force Fedora to reindex your content on startup by starting it with the following argument: -Dfcrepo.rebuild.on.start=true