...
- An account will have a unique identifier or token within the Bridge that corresponds with a set of credentials that provide access to the Bridge API
- Each account will be created/initialized by the DDP operating the bridge and the credentials needed to access the account will be provided to the account owner
- There will be a one-to-one relationship between a Bridge account and a single repository (or other) system
- The repository (or other) system accessing the Bridge will have an internally consistent method for ensuring that File IDs provided to the bridge uniquely identify individual bitstreams presented to the bridge for storage
- The Bridge application will ensure that each account is provided an independent namespace for File IDs
To Be Determined
- Method for authentication and authorization. Also need to determine if the same auth methods will be used by the Bridge and the Gateway
...
- Provides information about the bridge application. Can also be used to verify that the bridge application is available at the expected URL.
- Request:
GET /bridge
Response Body: JSON
Code Block { "bridge-version" : "", "supported-checksum-types" : "" # Possible values: MD5, SHA-256, SHA-512 }
- Response Code: 200 (on success)
...
- Purpose: Allows a repository with an OTM API to register itself with the Bridge. The information provided in this registration will allow the Bridge to make calls back to the OTM Gateway, such as to pull content listed in a deposit request. This call must be made prior to calls to deposit content. This call may also be made to update the Gateway information.
- Request:
POST /bridge/register
Request Body: JSON
Code Block language js { "gateway-url" : "", # Endpoint URL where the Bridge can call back to this OTM API "gateway-username" : "", # Credentials to allow the Bridge to make calls back into the OTM API "gateway-password" : "" # Credentials to allow the Bridge to make calls back into the OTM API }
- Response Code: 200 (on success)
- Notes:
- In the event that Gateway information is updated by calling this endpoint again, will those changes be applied to existing or in-process deposit requests?
- In order to make this call, an account with associated credentials will already need to exist in the Bridge. This will have been created in advance by the DDP (see Add Account below).
Deposit Content
- Purpose: Allows the repository to send content to the Bridge for deposit into the DDP.
- Request:
POST /bridge/deposit ? {checksum-type}
checksum-type
: (Optional) Applies to all file checksums (can be one of: MD5, SHA-256, SHA-512). Default is MD5.deposit-format
: (Optional) Format of content being deposited
Request Body: Request Body: JSON
Code Block language js { "filegroup-1-id" : { # filegroup is a generic grouping of files that can be used to capture structure such as in a digital object or work "version" : "", "files" : { "file-1-id" : "file-1-checksum", "file-2-id" : "file-2-checksum" } }, "filegroup-2-id" : { "file-3-idversion" : "file-1-checksum", "file-4-idfiles" : "file-2-checksum"{ "file-3-id" : "file-3-checksum", "file-4-id" : "file-4-checksum" } } }
- Response Code: 201 (on success)
Response BodyNotes: JSON
Code Block { "deposit-id" : "" }
- Each filegroup in the request body is considered an independent deposit which can be referred to in subsequent requests by the filegroup identifier.
- There is an explicit expectation that the content to be deposited will be available from the OTM Gateway at the path: "gateway-url" + / "filegroup-id" +/ + "file-id". How the Gateway resolves the file stream based on a request to this URL is irrelevant to the Bridge.
- Provided Filegroup IDs and File IDs must be URL-safe to support the Bridge requesting those files from the OTM Gateway and the reverse in the restore context
There is no guarantee that all files filegroups in a single deposit request will be deposited into the DDP at the same time. This allows the Bridge to manage transfers based on available resources (so as to not over-run local disk, for example).
- This does not currently allow for deposits where files use differing checksum types. Is this a necessary use case?
- This does not currently provide a method for grouping files (e.g. to specify all files in a "work"). Is there a need to capture groupings within the Bridge or DDP?
List Deposits
- The Bridge and DDP will consider the version value as opaque and not attempt to assign any meaning to the value.
- Not including "version" in the JSON is acceptable and is functionally the same as "version" : "". This "empty" version is equivalent to any other version.
- Attempting to deposit a filegroup with a filegroup ID and version that both match a previous deposit will be considered an error and rejected
- When new filegroup versions are deposited, the Bridge may choose to retrieve and submit to the DDP only those files which have changed between versions (based on file ID and checksum).
- The deposit-format parameter allows the depositor to specify the format of the content being deposited. The Bridge implementation may only support deposits of certain types.
- Agreements between the depositor and the DDP should specify which types are to be used for deposit.
- The Bridge and DDP will consider the version value as opaque and not attempt to assign any meaning to the value.
List Deposits
- Purpose: Retrieves a listing of all in-process deposits
- Request:
GET /deposit ? {status}
status:
(Optional) Limit list to deposits in a specific status
Response Body: JSON
Code Block { "filegroup-1-id" : { "version
- Purpose: Retrieves a listing of all in-process deposits
- Request:
GET /bridge/deposit ? {status}
status:
(Optional) Limit list to deposits in a specific status
Response Body: JSON
Code Block { "deposit-id-1" : { "filegroups" : "" # Number of filegroups in deposit "files" : "", # #Version Numberidentifier of files in depositfilegroup "statusfiles" : "", # Number of files in deposit "status" : "" # Current deposit status }, # Additional depositsfilegroups listed here }
- Response Code: 200 (on success)
- Notes:
- List requests made by a depositor will return only those deposits which were initiated by that depositing entity
- List requests made by Used by both the OTM Gateway (to check on submitted deposits) and the DDP (to determine if there are deposits which are ready for processing) will return results for all depositors
- It may be useful to include a depositor query parameter to allow the DDP to limit the response based on depositor
- This list will grow large if many deposits are initiated at once and may require paged results
- This list will not include all historical deposits, but The deposits returned by this method are only those that are in -process (possibly including those that have recently completed). There is no expectation that information about deposits that have completed successfully or were aborted will be available as part of this call indefinitely, as the deposits themselves are not intended to be the item of record. Information about files in completed deposits can be found by requesting an audit history report (below). Are there use cases that would require longer-term access to completed deposits via this endpoint?
Get Deposit Status
- the process of being deposited
Get Deposit Status
- Purpose: Allows the repository to ask for status of a given deposit
- Request:
GET /deposit/{filegroup
- Purpose: Allows the repository to ask for status of a given deposit
- Request:
GET /bridge/deposit/{deposit-id}
Response Body: JSON
Code Block { "depositfilegroup-id" : "",{ "status" : "deposit-state" : "", "files" #: Value{ based on defined set of known status states "file-1-id" : { "status", # Theredeposit couldstatus, beone moreof: informationpending, hereretrieved, existing like an approximate percentage completion of the current step, if known }
- Response Codes:
- 200 (on success)
- 404 (if the provided Deposit ID does not exist in the system)
- Notes:
- It would be possible to retain enough information to be able to provide a tombstone-type response for deposit IDs which were at some point in the system but have been removed
Abort Deposit
- Purpose: Allows the repository to ask the Bridge to abort the deposit process Request: DELETE /bridge/deposit/{deposit-id
"size" : "", # file size in bytes "checksum" : "" # file checksum }, "file-2-id" : { "status" : "", "size" : "", "checksum" : "" }, } } }
- Response Codes:
- 200 (on success)
- 404 (if the provided Deposit ID does not exist in the system)
- 403 (if the Deposit is not in a state which allows an Abort action to be applied)
- Notes:
- Only valid when deposits are in certain states (to be better defined as those states are clarified). An Abort will not be allowed if it cannot be performed cleanly by the Bridge
Response Body: JSON
Code Block |
---|
{
"details" : "", # In the event of a rejected call, information about why the call could not be performed will be provided here
} |
Restart Deposit
- Purpose: Allows the repository to ask the Bridge to re-start a deposit that failed. For example, if the repository became unavailable shortly after starting a deposit and the Bridge process failed due to this, or if a single file in the deposit was not in the repository when the Bridge attempted to retrieve it.
- Request:
POST /bridge/deposit/{deposit-id}/restart
- Response Code: 202 (on success)
- Notes:
- Only valid when deposits are in certain failure states (to be better defined as those states are clarified)
Delete Content
- Notes:
- It would be possible to retain enough information to be able to provide a tombstone-type response for filegroups which have completed deposit.
- Alternatively, requests for filegroups that have completed deposit could be forwarded to the Get Deposited endpoint.
- Deposit status
- Pending - File is waiting to be processed (it has not been retrieved)
- Retrieved - File has been successfully transferred to storage manage by the bridge
- Existing - File with a matching ID and checksum was part of a previous version that is already stored; this file will not be retrieved
- It would be possible to retain enough information to be able to provide a tombstone-type response for filegroups which have completed deposit.
Delete Content
- Purpose: Removes one or more files from DDP storage
- Request:
POST /delete ? {checksum-type}
# Using POST - Purpose: Removes one or more files from DDP storage
- Request:
POST /bridge/delete ? {checksum-type}
# Using POST rather than DELETE, allows for deleting multiple files (and is consistent with Restore action)checksum-type
: (Optional) if provided, applies to all file checksums (can be one of: MD5, SHA-256, SHA-512). Default is MD5.
Request Body: JSON
Code Block { "filefilegroup-1-id" : { "version" : "", "files" : { "file-1-id" : "file-1-checksum", # Checksum is optional, can be included to verify correct file is being deleted "file-2-id" : "file-2-checksum" }
- Response Code: 202 (on success)
} }, # Additional filegroups listed here }
- Response Code: 202 (on success)
Response Body: JSON
Code Block { "delete-id" : "" }
- Notes:
Response Body: JSON
Notes:Code Block { "delete-id" : "" }
In the versioning scenario, checksum may be used to select which version to delete. If no checksum is provided can either assume most recent version or all versions.- It would be possible to retain enough information to be able to provide a tombstone-type response for File IDs which were at some point in the system but have been removed.
- It would be possible to allow the "files" section to be omitted if all files in a given filegroup version are to be deleted.
- It would be possible to allow the value associated with a filegroup ID to be omitted if all files in all versions of a filegroup are to be restored.Is there any reason/need to specify the Filegroup for each file to be deleted?
List Deletes
- Purpose: Retrieves a listing of all in-process deletes
- Request:
GET /bridge/delete ? {status}
status:
(Optional) Limit list to deletes actions with a specific status
Response Body: JSON
Code Block { "delete-id-1" : { "files" : "", # Number of files in delete action "status" : "" # Current delete status }, # Additional delete actions listed here }
- Response Code: 200 (on success)
- Notes:
- Used by both the OTM Gateway (to check on delete requests) and the DDP (to determine if there are delete requests which are ready for processing)
- Similar to deposits, delete Delete IDs returned by this method are only those that are in-process.
...
- Purpose: Allows the repository to ask for status of a content delete action
- Request:
GET /bridge /delete/{delete-id}
Response Body: JSON
Code Block { "status" : "" } # This could include a top level status or a per-file status (or both)
- Response Code: 200 (on success)
- Notes:
- Need to better define the information that would be provided in the status response
Abort Delete - ?
Restart Delete - ?
List Deposited Content
List Deposited
- Purpose: Retrieves a list all deposited filegroup IDs
- Request:
GET /list
Response Body
- Purpose: Retrieve a list of all deposited files. There are 4 list options, (1) List all filegroups and associated files (2) List all filegroups, no files (3) List all files, no filegroups (4) List only files for the specified filegroup
- Request:
GET /bridge/list ? {list-type} & {filegroup}
list-type
: (Optional) Valid values are "all", "filegroups", and "files". Default is "all"filegroup
: (Optional) Indicates that only files for this specific filegroup should be listed
Response Body: JSON
Code Block language js { "filegroup-1-id" : { # filegroup is a generic grouping of files that can be used to capture structure such as in a digital object or work "file-1-id", "file-2-id", }, , "filegroup-2-id", ... }
Notes:
This list would not include deleted filegroups
Get Deposited Details
- Purpose: Retrieves details about a deposited filesgroup, including versions and associated files
- Request:
GET /list/{filegroup-id}/{file-id}
filegroup-id
: The identifier of the filegroup for which information is requestedfile-id
: (Optional) The identifier of the file for which information is requested
Response Body: JSON
Code Block language js { "checksum-type" : "" # Indicates type of checksums listed below, can be one of: MD5, SHA-256, SHA-512 "filegroup-1-id" : [ # filegroup is a generic #grouping Additionalof filegroupsfiles listed here }
or
Code Block { "filegroup-1-id", "filegroup-2-id", ... }
or
Code Block { "file-1-id", "file-2-id", ... }
- Notes
- There are varying options here. May need to consider the actual use cases and needs to see if all of these options are required
Restore Content
- Purpose: Requests that content be made available for restore
- Request:
POST /bridge/restore ? {checksum-type}
checksum-type
: (Optional) if provided, applies to all file checksums (can be one of: MD5, SHA-256, SHA-512). Default is MD5.
Request Body: JSON
Code Block { "file-1-id" : "file-1-checksum", # Checksum is optional, can be included to verify correct file is being restored that can be used to capture structure such as in a digital object or work { "version" : "", # filegroup version "files" : { "file-1-id" : { "size" : "", # file size in bytes "checksum" : "" # file checksum }, "file-2-id" : "file-2-checksum" }
Response Code: 202 (on success)
Response Body: JSON
Code Block { "restore-id{ "size" : "" }
- Notes:
- In the versioning scenario, checksum may be used to select which version to restore. If no checksum is provided most recent version is assumed.
- Is there any reason/need to specify the Filegroup for each file to be restored?
List Restores - ?
Restore Status
...
Response Body: JSON
Code Block |
---|
{ "status" : "" } # This could include a top level status or a per-file status (or both) |
, # file size in bytes "checksum" : "" # file checksum }, # Additional files listed here } }, # Additional filegroup versions listed here ] }
Notes:
- When a file-id is included, the file matching that ID is the only one returned in the response
This list would not include deleted versions or files. A tombstone capability is possible, but not currently part of this specification.
Restore Content
- Purpose: Requests that content be made available for restore
- Request:
POST /restore ? {checksum-type}
checksum-type
: (Optional) if provided, applies to all file checksums (can be one of: MD5, SHA-256, SHA-512). Default is MD5.
Request Body: JSON
Code Block { "filegroup-1-id" : { "version" : "", "files: { "file-1-id" : "file-1-checksum", # Checksum is optional, can be included to verify correct file is being restored "file-2-id" : "file-2-checksum" } }, # Additional filegroups listed here }
Response Code: 202 (on success)
Response Body: JSON
Code Block { "restore-id" : "" }
- Notes:
- In the versioning scenario, checksum may be used to select which version to restore. If no checksum is provided most recent version is assumed.
- It would be possible to allow the "files" section to be omitted if all files in a given filegroup version are to be restored.
- It would be possible to allow the value associated with a filegroup ID to be omitted if all files in all versions of a filegroup are to be restored.
List Restores
- Purpose: Retrieves a listing of all in-process restores
- Request:
GET /restore ? {status}
status:
(Optional) Limit list to restores actions with a specific status
Response Body: JSON
Code Block { "restore-id-1" : { "files" : "", # List of files in restore action "status" : "", # Current restore status "expiration" : "" # Date on which restored content will no longer be available }, # Additional restore actions listed here }
- Response Code: 200 (on success)
- Notes:
- Used by both the OTM Gateway (to check on restore requests) and the DDP (to determine if there are restore requests which are ready for processing)
- Restore IDs returned by this method are only those that are in-process.
Restore Status
- Purpose: Allows the repository to ask for status of a specific content restore action
- Request:
GET /restore/{restore-id}
Response Body: JSON
Code Block { "files" : "", # List of files in restore action "status" : "" # This could include a top level status or a per-file status (or both) "expiration" : "" # Date on which restored content will no longer be available }
- Response Code: 200 (on success)
- Notes:
- Restored content will only be made available on the Bridge for a limited time. After this time, content will be removed and the status of the restore will be changed to expired.
Get Restored Content
- Purpose: Allows the repository to retrieve restored content from the Bridge
- Request:
GET /restore/{restore-id}/{filegroup-id}/{file-id} ? {checksum-type}
filegroup-id:
The identifier of the filegroup for the restored filefile-id:
The identifier of the restored file to retrievechecksum-type
: (Optional) Defines the type of checksum to be included in the response ETag header. Can be one of: MD5, SHA-256, SHA-512. Default is MD5.
Response: Restored file content stream
- Response Codes:
- 200 (on success)
- 404 (if the file is not part of the restore or the restored content has expired)
- Response Headers: ETag
Get Audit Log
- Purpose: Retrieves an audit history report. The report will list audit events associated with a single file or set of files in a filegroup.
- Request:
GET /audit/{filegroup-id}/{file-id}
filegroup-id:
The identifier of the filegroup for which an audit report is requestedfile-id:
(Optional) The identifier of the file for which an audit report is requested. If this is not provided the audit report is requested based on the filegroup-id.
- Response Codes:
- 200 (on success)
- 404 (if there is no audit information for the provided ID)
Response Body: JSON
Code Block { "filegroup-id" : [ { "file-id" : [ { "date" : "", "type" : "", "event" : "" }, # Additional events for this file listed here ], # Additional files listed here } ] }
- Notes:
- It may be desirable to provide paged results for large result sets
- Types and fields of expected history events are to be listed in an appendix
Bridge API - Endpoints to be used by DDP
Add Account
- Purpose: Allows the DDP to create a new depositing user account in the Bridge. After this call it will be possible for the OTM Gateway to register with the Bridge. This call can also be used to reset the access credentials for an account.
- Request:
POST /account/{account-name}
account-name:
The name associated with an account
- Response Code: 201 (on success)
Response Body: JSON
Code Block language js { "account-name" : "", # Name of the account "account-username" : "", # Credentials to allow calls to the Bridge for this account "account-password" : "" # Credentials to allow calls to the Bridge for this account }
List Accounts
- Purpose: Allows the DDP to list all accounts known by the Bridge
- Request:
GET /account
- Response Code: 200 (on success)
Response Body: JSON
Code Block language js { "account-1-name", "account-2-name" }
Complete Deposit
...
- It may be useful to provide a listing of the files that can be retrieved based on this restore request in the response (not assuming that the original requester of the restore kept that list around from the initial call).
- Restored content will only be made available on the Bridge for a limited time. After this time, content will be removed and the status of the restore will be changed to expired.
Abort Restore - ?
Restart Restore - ?
Get Restored Content
- Purpose: Allows the repository to retrieve restored content from the Bridge
- Request:
GET /bridge/restore/{restore-id}/{file-id} ? {checksum-type}
checksum-type
: (Optional) Defines the type of checksum to be included in the response ETag header. Can be one of: MD5, SHA-256, SHA-512. Default is MD5.
Response: Restored file content stream
- Response Codes:
- 200 (on success)
- 404 (if the file is not part of the restore or the restored content has expired)
- Response Headers: ETag
Request Audit History Report
...
- id-type: (Optional) Specifies the type of ID provided in the request body. All IDs must be of the same type. Allowed values are: filegroup, file, deposit, restore, or delete. Default value is file.
Request Body: JSON
Code Block |
---|
{ "id-1", "id-2", ... } |
...
Response Body: JSON
Code Block |
---|
{
"audit-report-id" : ""
} |
Get Audit History Report
...
Response Body: JSON
Code Block |
---|
{
"file-1-id" : {
"audit-event-1",
"audit-event-2",
},
} |
...
- 200 (on success)
- 404 (if there is no audit report with the provided ID)
...
- Events associated with a deposit, restore, or delete should include the appropriate deposit, restore, or delete ID as part of the audit event
- A format for audit events will need to be defined to support processing of these events.
- Audit reports will be retained by the Bridge for some pre-defined length of time (TBD), after which they will be purged and a new report request would be required
Bridge API - Endpoints to be used by DDP
...
- Purpose: Allows the DDP to inform the Bridge of a new OTM account. After this call it will be possible for the OTM endpoint to contact the Bridge.
...
- that a deposit has completed.
- Request:
POST /deposit/{filegroup-id}
- Response Code: 200 (on success)
Complete Delete
- Purpose: Allows the DDP to inform the Bridge that a deposit has transitioned to a new status. Certain status updates (such as when a deposit is completed) may trigger additional actions within the Bridge.
...
- delete has completed.
- Request:
POST /delete/{delete-id}
- Response Code: 200 (on success)
Complete Restore
- Purpose: Allows the DDP to inform the Bridge that a restore has transitioned to a new status. Certain status updates may trigger additional actions within the Bridge.
...
- completed.
- Request:
POST /restore/{restore-id}
- Response Code: 200 (on success)
Add Audit Event
- Purpose: Update audit index by providing additional audit details
DDP API - Endpoints to be used by the Bridge
Content Ready for Deposit
- Request:
POST /audit/{id}
id:
The identifier for the file or filegroup to which this audit event is to be applied
Request Body: JSON
Code Block { "audit-event" }
- Response Code: 200 (on success)
- Notes:
- Rather than a direct call from the DDP to the Bridge, this could be implemented as a notification stream to which the Bridge is listening.
Purpose: A set of content is available for deposit- Dropping this call based on the assumption that the DDP will query the Bridge to determine when new deposits are ready (rather than the Bridge notifying the DDP)