Checksum Checker

Checksum Checker is program that can run to verify the checksum of every item within DSpace. Checksum Checker was designed with the idea that most System Administrators will run it from the cron. Depending on the size of the repository choose the options wisely.

Command used:

[dspace]/bin/dspace checker

Java class:

org.dspace.app.checker.ChecksumChecker

Arguments short and (long) forms):

Description

-L or --continuous

Loop continuously through the bitstreams

-a or --handle

Specify a handle to check

-b <bitstream-ids>

Space separated list of bitstream IDs

-c or --count

Check count

-d or --duration

Checking duration

-h or --help

Calls online help

-l or --looping

Loop once through bitstreams

-p <prune>

Prune old results (optionally using specified properties file for configuration

-v or --verbose

Report all processing

There are three aspects of the Checksum Checker's operation that can be configured:

Checker Execution Mode

Execution mode can be configured using command line options. Information on the options are found in the previous table above. The different modes are described below.

Unless a particular bitstream or handle is specified, the Checksum Checker will always check bitstreams in order of the least recently checked bitstream. (Note that this means that the most recently ingested bitstreams will be the last ones checked by the Checksum Checker.)

Available command line options

Checker Results Pruning

As stated above in "Pruning mode", the checksum_history table can get rather large, and that running the checker with the -p assists in the size of the checksum_history being kept manageable. The amount of time for which results are retained in the checksum_history table can be modified by one of two methods:

  1. Editing the retention policies in [dspace]/config/dspace.cfg See Chapter 5 Configuration for the property keys. OR
  2. Pass in a properties file containing retention policies when using the -p option.To do this, create a file with the following two property keys:

    checker.retention.default = 10y
    checker.retention.CHECKSUM_MATCH = 8w

    You can use the table above for your time units. At the command line: [dspace]/bin/dspace checker -p retention_file_name <ENTER>

Checker Reporting

Checksum Checker uses log4j to report its results. By default it will report to a log called [dspace]/log/checker.log, and it will report only on bitstreams for which the newly calculated checksum does not match the stored checksum. To report on all bitstreams checked regardless of outcome, use the -v (verbose) command line option:

[dspace]/bin/dspace checker -l -v (This will loop through the repository once and report in detail about every bitstream checked.

To change the location of the log, or to modify the prefix used on each line of output, edit the [dspace]/config/templates/log4j.properties file and run [dspace]/bin/install_configs.

Cron or Automatic Execution of Checksum Checker

You should schedule the Checksum Checker to run automatically, based on how frequently you backup your DSpace instance (and how long you keep those backups). The size of your repository is also a factor. For very large repositories, you may need to schedule it to run for an hour (e.g. -d 1h option) each evening to ensure it makes it through your entire repository within a week or so. Smaller repositories can likely get by with just running it weekly.

Unix, Linux, or MAC OS. You can schedule it by adding a cron entry similar to the following to the crontab for the user who installed DSpace:

0 4 * * 0 [dspace]/bin/dspace checker -d2h -p

The above cron entry would schedule the checker to run the checker every Sunday at 400 (4:00 a.m.) for 2 hours. It also specifies to 'prune' the database based on the retention settings in dspace.cfg.

Windows OS. You will be unable to use the checker shell script. Instead, you should use Windows Schedule Tasks to schedule the following command to run at the appropriate times:

[dspace]/bin/dspace checker -d2h -p

(This command should appear on a single line).

Automated Checksum Checkers' Results

Optionally, you may choose to receive automated emails listing the Checksum Checkers' results to the email address specified in the mail.admin configuration property. Schedule it to run after the Checksum Checker has completed its processing (otherwise the email may not contain all the results).

Command used:

[dspace]/bin/dspace checker-emailer

Java class:

org.dspace.checker.DailyReportEmailer

Arguments short and (long) forms):

Description

-a or --All

Send all the results (everything specified below)

-d or --Deleted

Send E-mail report for all bitstreams set as deleted for today.

-m or --Missing

Send E-mail report for all bitstreams not found in assetstore for today.

-c or --Changed

Send E-mail report for all bitstreams where checksum has been changed for today.

-u or --Unchanged

Send the Unchecked bitstream report.

-n or --Not Processed

Send E-mail report for all bitstreams set to longer be processed for today.

-h or --help

Help

You can also combine options (e.g. -m -c) for combined reports.

Cron. Follow the same steps above as you would running checker in cron. Change the time but match the regularity. Remember to schedule this after Checksum Checker has run.