Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

By default, files larger than 4KB are stored on disk named after their SHA1 digest, in the directory fcrepo.binary.directory. (4KB is the default, but can be changed by updating the minimumBinarySizeInBytes property in repository.json).  That is, a file with the SHA1 "a1b2c369563c0465ab82cdb2789d45ce1c3585b1" would be stored on disk in /path/to/fcrepo4-data/fcrepo.binary.directory/a1/b2/c3/a1b2c369563c0465ab82cdb2789d45ce1c3585b1.  So files in the repository can be backed up backing up the directory fcrepo.binary.directory.

LevelDB Backup

LevelDB stores it's data as flat files in the directory fcrepo.ispn.repo.cache of the fcrepo home. The fcrepo home directory can be backed up as a whole to create a snapshot of the repository with both the binaries and the metadata. Though, the fcrepo.binary.directory and fcrepo.ispn.repo.cache are the only directories necessary for backup. (See ModeShape Artifacts Layout).  The backup can be created on a live repository without having to shutdown or restricting ingests to the repository. Though, it would be good idea to schedule the backups after any batch ingests, so that the newly ingested data is also included in the backup. 

Backup Strategies

Here are a few strategies for backup:

WITH SHUTTING DOWN FEDORA (CONSISTENTLY RELIABLE BACKUPS

STEPS:

  1. Shutdown Fedora
  2. Backup of FCREPO HOME (or just fcrepo.binary.directory and fcrepo.ispn.repo.cache)
  3. Restart Fedora

WITH PAUSING WRITES TO FEDORA

STEPS:

  1. Pause all updates to the repository.
    1. Do not create, delete, or update OBJECTS or DATASTREAMS.
  2. Wait for all previous update requests to be processed.
    1. For serialization of newly created objects to complete.
    2. And, for leveldb background compaction to complete (usually in seconds), if the previous updates triggered a compaction.
  3. Backup of FCREPO HOME (or just fcrepo.binary.directory and fcrepo.ispn.repo.cache)
    1. Verify successful backup.
  4. Continue with the updates.

HOT BACKUPS (LESS RELIABLE UNLESS VERFIED)

  1. Backup of FCREPO HOME (or just fcrepo.binary.directory and fcrepo.ispn.repo.cache)
    1. Verify successful backup.
Verifying Backups:
  1. Check leveldb cache store directory from the backup using a leveldb client.
    1. Verify the leveldb opens.
    2. Verify by iterating through the keys. (To expose any corruption)