Description
There are two types of Storage which we track in Chronopolis. The first is the StorageRegion, which defines an area at a Chronopolis Node which can be used for Intake services. The second is a StagingStorage object which defines each usage of the StorageRegion by an Intake service.
Storage Regions
At the moment Storage Regions are primarily used for Intake Services which will specify which region a Bag resides in during a deposit. Then, depending on the StorageRegion used, different replication information is supplied when creating Replications which is useful when we are replicating off of multiple servers and data centers.
Currently there are two types of Storage Regions - BAG and TOKENS. This was done in order to allow the Intake Server to continue to write tokens out to a filesystem for replication.
Adding and Updating StorageRegions
~~example and documentation~~
Staging Storage
Staging Storage objects are primarily used by the Intake Services in order to determine where data lives and how much is used. In addition, upon completion of all Replications, an Intake Service must clean up after itself and deactivate the Staging Storage object in the Ingest Server. Further usage of Staging Storage objects can be found in the Bag documentation.
See Also:
API Documentation
Documentation about StorageRegions, StagingStorage, and Fixity and their purpose in the Ingest Server
...
Storage Region
A region where staged data may live
| Field | Type | Description |
|---|---|---|
| id | Long | The id of a StorageRegion in the database |
| node | String | The namespace of the node which uses a StorageRegion |
| note | String | A note accompanying the StorageRegion to further identify it |
| capacity | Long | The amount of capacity which an Intake Service is allotted for a StorageRegion |
| dataType | DataType | The type of data to be stored on a StorageRegion |
| storageType | StorageType | The type of storage of the StorageRegion |
| replicationConfig | ReplicationConfig | The configuration information used when creating Replications for a StorageRegion |
Replication Config
| Field | Type | Description |
|---|---|---|
| region | Long | The StorageRegion associated with the ReplicationConfiguration |
| path | String | The path to use when creating a Replication |
| server | String | The fully qualified domain name of the server which clients will connect to |
| username | String | The user which a client should connect as; if left empty it will default to the clients node namespace |
Staging Storage
Staging Information about where content is stored and fixity information for that data
| Field | Type | Description |
|---|---|---|
| size | Long | The total size of the staged data |
| path | String | The relative path to the staged data (can point to either a file or a directory) |
| region | Long | ID of the StorageRegion which the data is staged on |
| active | Boolean | Boolean flag showing if the data is still staged |
| totalFiles | Long | The number of files staged |
| fixities | Set<Fixity> | Fixity values of the staged data |
Fixity
Fixity Information for validating staged content
| Field | Type | Description |
|---|---|---|
| value | String | The computed digest |
| algorithm | String | The name of the algorithm used |
| createdAt | DateTime | The time of creation of the digest |
DataType
An enumeration of possible Data which can be stored on a StorageRegion
...
chron.stage.tokens.posix.{id,path}Staging Storage
The StagingStorage entity is used in the Ingest Server to track when a Bag has been staged for distribution. It contains a flag which marks if it is active, which can be used to determine if the space it uses on a filesystem may be freed. The Fixity information alongside the StagingStorage is used in order to validate any Replication which occurs. At the moment, for a Bag the tagmanifest is used and for a Token the TokenStorage is used.
...