...
About This Page
The purpose of this page is to inform the design and development of Akubra by providing an analysis of existing Blob Storage APIs and implementations.
...
- Be concerned with container (directory/folder/collection) semantics.
- Be concerned with versioning semantics (e.g. revision history).
APIs and Implementations
API | Language | Target Application | Known Implementations |
---|---|---|---|
Java | General-Purpose | AKU-FS: FSBlobStore | |
Java | Alfresco CMS | ALF-FS: FileContentStore | |
Java | DSpace 2 (Unreleased) | BIT-FED: FedoraBitStore | |
Python | Any Using Django | ||
Java | Daisy CMS | ||
COM | MS SQL Server | ||
Java | Apache JackRabbit | ||
Java | Apache JackRabbit | ||
Java | Fedora Repository (2.x to 3.1) | LLS-FS: GenericFileSystem | |
Java | Topaz OTM | ||
Java / C | General-Purpose |
Capability Matrix
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
---|---|---|---|---|---|---|---|---|---|---|---|
AKU-API | URI | Internal or External | Yes | Yes | Size | ? | All | Optional, JTA | |||
AKU-FS | file:/// URI | Internal | Yes | Yes | Size | None | All | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
ALF-API | URL | Internal or External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-FS | store:// URL | Internal or External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-HTT | http:// URL | N/A | No | No | Size | None | None | Optional, JTA | |||
ALF-REP | URL | Internal or External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-ROU | URL | Internal or External | Yes | Yes | Size | None | All | Optional, JTA | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
BIT-API | Unicode String | Internal | Yes | Yes | Any (Map) | None | None | No | |||
BIT-FED | (NCNameChar)* | Internal | Yes | No | Size | None | None | No | |||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="020f0ada-6d28-4b24-bcdb-14b15db9afae"><ac:plain-text-body><![CDATA[ | BIT-FS | ([DEV:0-9]){38,40} | Internal | Yes | Yes | Size | None | None | No <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="2dd80693-d2ba-4614-b1c8-1bc24f9efff7"><ac:plain-text-body><![CDATA[ | ||
BIT-HON | ([DEV:0-9]){38,40} | Internal | Yes | Yes | ? | None | None | No | |||
]]></ac:plain-text-body></ac:structured-macro> | <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="241bc9d9-a108-4a8e-8406-19bf831e9baf"><ac:plain-text-body><![CDATA[ | BIT-S3 | ([DEV:0-9]){38,40} | Internal | Yes | Yes | Size | None | None | No | <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="1392b6c4-49d7-40e7-bfe5-f14fda954fcd"><ac:plain-text-body><![CDATA[ |
BIT-SRB | ([DEV:0-9]){38,40} | Internal | Yes | Yes | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
DJA-API | Unicode String | External | Yes | Yes | Size | None | None | No | |||
DJA-FS | Unicode String | External | Yes | Yes | Size | None | None | No | |||
DJA-S3 | Unicode String | External | Yes | Yes | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
DSY-API | Unicode String | Internal | Yes | No | None | None | None | No | |||
DSY-FS | Unicode String | Internal | Yes | No | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="18d36354-4754-4009-8ed7-410438b248b8"><ac:plain-text-body><![CDATA[|||
EBS-API | byte[] | Internal | Yes | No | Size | None | None | No | |||
EBS-FS | GUID or String | Internal | Yes | No | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
JKB-API | Unicode String | Internal | Yes | Yes | None | None | None | No | |||
JKB-DB | propertyId + index | Internal | Yes | Yes | None | None | None | No | |||
JKB-FS | Filesystem Path | Internal | Yes | Yes | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
JKD-API | Hex String | Internal | Yes | No | Size | None | All | No | |||
JKD-DB | SHA-1 Digest of Content | Internal | Yes | No | Size | None | All | No | |||
JKD-FS | SHA-1 Digest of Content | Internal | Yes | No | Size | None | All | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
LLS-API | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-FS | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-HON | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-IRD | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-S3 | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-SRB1 | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
LLS-SRB2 | pid or pid + dsId | External | Yes | Yes | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
OTM-API | Unicode String | External | Yes | No | None | None | None | Optional, JTA | |||
OTM-FED | Unicode String | External | Yes | No | None | None | None | Best-effort, JTA | |||
OTM-FS | Unicode String | External | Yes | No | None | None | None | Yes, JTA | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
XAM-API | XUID | Internal | Yes | No 1 | Last-Modified | Several Predefined | All | No | |||
XAM-CEN | XUID | Internal | Yes | No | Last-Modified | Several Predefined | All | No | |||
XAM-HON | XUID | Internal | Yes | No | Last-Modified | Several Predefined | All | No | |||
XAM-IAP | XUID | Internal | Yes | No | Last-Modified | Several Predefined | All | No |
1 XAM is designed with a fixed-content assumption – - data cannot be modified without changing its XUID.
2 In XAM, all metadata fields are either binding or non-binding. Binding metadata cannot change while keeping the same XUID for an XSet, while non-binding metadata can.
Other APIs/Protocols/Impls of Interest
- Apache Sling - REST interface on JCR, need to look at this more...
- AtomPub - Collection semantics
- Azure Blob Storage API - Collection semantics, REST-based
- CMIS - Draft; has versioning and container semantics.
- DSpace 1 BitstreamStorageManager - Single impl, supports either local filesystem or SRB
- Eclipse - BlobStore- Single impl, maps UUIDs to files on disk
- Google Datastore API - Google's "cloud" underneath, REST-ish, container semantics, transactions!...but quite limited today (1MB limit on content, 1000 limit on local files...etc)
- JAF DataSource - Not a Store API; just models a file with a MIME type
- JSR170/283 - JCR; has versioning and container semantics.
- Jakarta Slide ContentStore - Defunct; has versioning semantics
- Jargon API for SRB and iRODS - Models files and directories, with metadata
- JSPWiki FileSystemProvider - Fedora Repository Impl available; API has versioning semantics.
- MediaWiki FileStore - PHP source
- MogileFS - Obtaining
- GlusterFS- a clustered file-system capable of scaling to several peta-bytes
- NFS - Directory semantics.
- OKI Repository OSID - Has container semantics (Assets may contain other assets)
- Plone Large File Management - Several options, but no common interfacealternatives to ZODB. Appear to be explicitly configured as part of a "content type". For example, see FileSystemStorage
- Sakai Content Hosting Service - Collection semantics. JCR impl
- Skwish - Transactions and reads/writes over a collection of "segments" on a filesystem. See MotivationMediaWiki FileStore - PHP source
- SMB/CIFS - Directory semantics.
- Storage Resource Manager - Used by LHC; does not provide direct data access, only references.