Reports

Data generated by your S3 client about your content will be saved in the -managed bucket associated with your S3 account.

After you begin creating buckets and uploading content, you will start seeing folders in the -managed bucket, including

audit: there will be a folder for each of the buckets you created; this data is mostly machine-readable and provides details about activities performed on your data.
exports: folders in this bucket will provide checksums data organized by date. Within each of those folders will be two sub-folders, one labeled AWSDynamoDB and another labeled CSV. We recommend ignoring the data in the AWSDynamoDB folder unless you are an advanced user, which is highly technical data generated by AWS, and focusing instead on the file manifests with individual checksums for each file in the CSV folder.
- Header columns in the CSV exports include
  - BucketName (where you uploaded content)
  - ObjectKey (filename)
  - Checksum (the file's checksum)
  - LastChecksumSuccess (will be either TRUE or FALSE)
  - LastChecksumDate
  - LastChecksumMessage ("ok" unless there was a checksum failure)

inventory: again, there will be a folder for each of the buckets you created; this folder structure provides a listing of all files in each bucket with a small amount of metadata about the files. Most of this data is technical; you likely will want to review the CSV data in the inventory/$bucket/inventory/csv folder structure.
- Header columns in these CSV exports include
  - BucketName (where you uploaded content)
  - ObjectKey (filename; this CSV includes any folder structure/directories you have uploaded to a bucket)
  - version_id (the latest ID of the object)
  - is_latest (TRUE or FALSE)
  - delete_marker (TRUE or FALSE; probably FALSE unless you have deleted the object)
  - size (size of the uploaded object in bytes; folders or directories will show 0 in size and deleted items will have an empty cell)
  - last_modified (timestamp)
  - storage_class (either STANDARD or GLACIER) - objects remain in standard storage for 7 days before moving to Glacier, unless they are in a -public bucket
- Also in the inventory folder is a stats folder, which will contain .json files from which the HTML reports are generated in the reports folder.
logs: .txt files regarding actions taken; check this folder for error messages when creating buckets and during other steps.
reports: Storage reports regarding your DuraCloud usage, saved as HTML files. These reports are the most human-readable and include number of files and storage size broken down by bucket, as well as your overall usage.

You can download data from all these folders for reviewing and local storage if you desire.

General tip

As a user, the data you're most likely interested in will be in the CSV files in the -exports bucket, the CSV inventories in the inventory/$bucket/inventory/csv filepath, and the HTML files in the -reports bucket. Logs will be most useful if you're encountering errors when creating buckets.

Cyberduck Tips

Logs and other items you wish to download for viewing will go straight to your Downloads folder. You can update this by going to Edit --> Preferences --> Transfers, under the General tab and choose a different folder if you wish.
Right-click (control+click on a Mac keyboard or 2-finger click on a touch pad) to change filenames when downloading manifests and other files for review. This will allow you to control your file naming structure and ensure similarly-named files aren't overwritten when downloading (or get warnings about overwriting files from Cyberduck).

Page tree

0 Comments