Versions Compared

Old Version 14

changes.mady.by.user Michael Ritter

Saved on

compared with

New Version 15

changes.mady.by.user Sibyl Schaefer

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Repairing is a process which allows for node to node resolution of corrupt files in Chronopolis. Currently, a node administrator must choose which files to repair and which node to repair from. This is to prevent unnecessary repairs (e.g. in the event of due to errors from a filesystem being offline) , and also to allow for discussion around the collection before it is repaired.

...

Installation Notes

The rpm creates a chronopolis Chronopolis user if it does not exist, and creates the following files/directories:

...

service chron-repair start|stop|restart

Workflow

Note that we have the workflow involves two nodes: one with CORRUPT data and one with VALID data

  1. CORRUPT notices data in their Audit Manager (ACE-AM) is showing File Corrupt, indicating that checksums on disk have changed
    1. Discussion happens internally about who has this and can repair it
    2. SSH keys exchanged so that data transfer can occur for files which are to be repaired
  2. CORRUPT logs on to the Ingest server and selects 'Request Repair' in order to create a 'Repair Request'
    1. Inputs ACE AM credentials to query for the corrupt collection
    2. Select the Collection
    3. Select the Files to repair and the Node where they will be Repaired
  3. VALID logs onto the Ingest server and selects 'Fulfill Repair' in order to stage data for the repair
  4. At this point, both CORRUPT and VALID nodes should start the Repair service
    1. The Repair service running at VALID will stage data and update the Repair
    2. The Repair service running at CORRUPT will
      1. Pull data from VALID into a staging area
      2. Validate that the data transferred and matches the checksums in the ACE AM
      3. Overwrite the corrupt files
      4. Audit the files in the ACE AM
      5. Update the Repair with the result of the audit
  5. Once complete, the Repair Service at each node can be stopped

...

  • Direct transfer
    • through rsync
    • through ACE AM
  • Indirect transfer through the Ingest Server

During development, support was added for each type of transfer, but only the direct rsync strategy was implemented. The direct ACE-AM transfer strategy requires additional development in the Audit Manager in order to support API Keys which can be used to access content. The indirect transfer through the Ingest Server was omitted as it was not deemed onerous for Chronopolis Nodes to exchange ssh keys.

...