Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Add fcrepo.rebuild.continue

Overview

Fedora maintains a set of internal indices containing information derived from the persistent storage, namely the underlying OCFL (Oxford Common File Layout). These indices act only as a cache of information that is persisted in the underlying OCFL and can be destroyed and rebuilt as necessary.

...

These indices are built initially when Fedora is started for the first time on a non-empty OCFL.  As content is added, updated and deleted, the indices are updated accordingly.   You may at some point wish to rebuild these indices from scratch for a variety of reasons.    For For example,  it is likely that a future change (in subsequent releases) to the index structures could require a rebuild.  In that case you can accomplish this task by taking simply restarting Fedora with the following steps.

  1. Stop Fedora 
  2. Remove the index data in your database (see instructions below for the database you are using)
  3. Start Fedora

flag:

Code Block
-Dfcrepo.rebuild.on.start=true

NOTE: Depending on the size of your repository, the type of database,  and/or the capacity of your database server,  the rebuild can take anywhere from a few seconds to several hours (i.e if you have many millions of objects). 

Removing Index Data

Step 2 above will be accomplished differently depending on which underlying database you're using and how your system is configured. 

Embedded (H2) Database users

NOTE: The H2 database should not be used in a production installation.

To remove the default (ie H2 database) simply delete the following files from your <fedora_home>/data directory if they exist:

  • fcrepo-h2.lock.db
  • fcrepo-h2.mv.db

Postgres, MariaDB, and MySQL Users

If you are using one of the three available SQL databases, you'll need to take one of the following actions:  

...

Drop or truncate the fcrepo database tables if it is inconvenient to drop the entire database. Below is a list of the affected tables.

...

Continuing a failed rebuild

If a previous rebuild of the Fedora indices was interrupted and you have some records indexed in the database you can "continue" the rebuild with the following flag:

Code Block
-Dfcrepo.rebuild.continue=true

This flag will restart the process of iterating over all objects in the OCFL filesystem but will only index objects it does not find in the existing indices. This flag takes precedence over the fcrepo.rebuild.on.start flag, if both are specified the indices will NOT be truncated.

Info

For the full list of database properties, please refer to the Database section of the Properties documentation.

You can find additional information on this wiki about configuring supported databases and the structure of the tables in the database.

Additional Configuration

To help with long rebuilds, optional properties can be passed in

PropertyDefault ValueNote
-Dfcrepo.rebuild.validationtrueEnables validation of the ocfl repository before running reindexing. Set to false to skip this step.
-Dfcrepo.rebuild.validation.fixityfalseSets a flag in the ocfl validation to run fixity checks on bitstreams during validation. Set to true to enable fixity checks.

Note: -Dfcrepo.rebuild.validation must also be enabled for fixity checking
-Dfcrepo.ocfl.verify.inventorytrueTells the ocfl repository that the inventory for an ocfl object should be validating during read and write operations. Set to false to disable checking of the inventory.