...
About This Page
The purpose of this page is to inform the design and development of Akubra by providing an analysis of existing Blob Storage APIs and implementations.
...
- Be concerned with container (directory/folder/collection) semantics.
- Be concerned with versioning semantics (e.g. revision history).
APIs and Implementations
API | Language | Target Application | Known Implementations |
---|---|---|---|
Java | General-Purpose | AKU-FS: FSBlobStore | |
Java | Alfresco CMS | ALF-FS: FileContentStore | |
Java | DSpace 2 (Unreleased) | BIT-FED: FedoraBitStore | |
Python | Any Using Django | ||
Java | Daisy CMS | ||
COM | MS SQL Server | ||
Java | Apache JackRabbit | ||
Java | Apache JackRabbit | ||
Java | Fedora Repository (2.x to 3.1) | LLS-FS: GenericFileSystem | |
Java | Topaz OTM | ||
Java / C | General-Purpose |
Capability Matrix
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
---|---|---|---|---|---|---|---|---|---|---|---|
AKU-API | URI | User Internal or Store External | Yes | Yes | Size | ? | All | Optional, JTA | |||
AKU-FS | file:/// URI | Store Internal | Yes | Yes | Size | None | All | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
ALF-API | URL | User Internal or Store External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-FS | store:// URL | User Internal or Store External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-HTT | http:// URL | N/A | No | No | Size | None | None | Optional, JTA | |||
ALF-REP | URL | User Internal or Store External | Yes | Yes | Size | None | All | Optional, JTA | |||
ALF-ROU | URL | User Internal or Store External | Yes | Yes | Size | None | All | Optional, JTA | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
BIT-API | Unicode String | User or Store Internal | Yes | Yes | Any (Map) | None | None | No | |||
BIT-FED | (NCNameChar)* | User or Store Internal | Yes | No | Size | None | None | No | |||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="90ac7e0b-50c3-4323-a2de-0a953514f504"><ac:plain-text-body><![CDATA[ | BIT-FS | ([DEV:0-9]){38,40} | User or Store Internal | Yes | Yes | Size | None | None | No | ||
BIT-HON | ? | ? | ? | ([DEV:0-9]){38,40} | Internal | Yes | Yes ? | ? | None | None | No <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="a33de1e6-077d-40e1-898d-31dcf641ea37"><ac:plain-text-body><![CDATA[ |
BIT-S3 | ([DEV:0-9]){38,40} | User or Store Internal | Yes | Yes | Size | None | None | No | <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="003c1d2b-2299-4831-bc51-3ad85c91cea8"><ac:plain-text-body><![CDATA[ | ||
BIT-SRB | ([DEV:0-9]){38,40} | User or Store Internal | Yes | Yes | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
DJA-API | Unicode String | User External | Yes | Yes | Size | None | None | No | |||
DJA-FS | Unicode String | User External | Yes | Yes | Size | None | None | No | |||
DJA-S3 | Unicode String | User External | Yes | Yes | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
DSY-API | Unicode String | Store Internal | Yes | No | None | None | None | No | |||
DSY-FS | Unicode String | Store Internal | Yes | No | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="7b5e594f-d5ff-4233-bb94-3d7b7c128188"><ac:plain-text-body><![CDATA[ | EBS-API | byte[] | Store Internal | Yes | No | Size | None | None | No | ||
EBS-FS | GUID or String | Store Internal | Yes | No | Size | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
JKB-API | Unicode String | Store Internal | Yes | Yes | None | None | None | No | |||
JKB-DB | propertyId + index | Store Internal | Yes | Yes | None | None | None | No | |||
JKB-FS | Filesystem Path | Store Internal | Yes | Yes | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
JKD-API | Hex String | Store Internal | Yes | No | Size | None | All | No | |||
JKD-DB | SHA-1 Digest of Content | Store Internal | Yes | No | Size | None | All | No | |||
JKD-FS | SHA-1 Digest of Content | Store Internal | Yes | No | Size | None | All | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
LLS-API | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-FS | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-HON | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-IRD | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-S3 | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-SRB1 | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
LLS-SRB2 | pid or pid + dsId | User External | Yes | Yes | None | None | None | No | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
OTM-API | Unicode String | User External | Yes | No | None | None | None | Optional, JTA | |||
OTM-FED | Unicode String | User External | Yes | No | None | None | None | Best-effort, JTA | |||
OTM-FS | Unicode String | User External | Yes | No | None | None | None | Yes, JTA | |||
API/Impl | ID Type | ID Gen | Write? | Rewrite? | Read-Only MD | Writable MD | Listing Options | Transactions | |||
XAM-API | XUID | Store Internal | Yes | No 1 | Last-Modified | Several Predefined | All | No | |||
XAM-CEN | XUID | Store Internal | Yes | No | Last-Modified | Several Predefined | All | No | |||
XAM-HON | XUID | Store Internal | Yes | No | Last-Modified | Several Predefined | All | No | |||
XAM-IAP | XUID | Store Internal | Yes | No | Last-Modified | Several Predefined | All | No |
1 XAM is designed with a fixed-content assumption – - data cannot be modified without changing its XUID.
2 In XAM, all metadata fields are either binding or non-binding. Binding metadata cannot change while keeping the same XUID for an XSet, while non-binding metadata can.
Other APIs/
...
Protocols/Impls of Interest
- Apache Sling - REST interface on JCR, need to look at this more...
- AtomPub - Collection semantics
- Azure Blob Storage API - Collection semantics, REST-based
- CMIS - Draft; has versioning and container semantics.
- DSpace 1 BitstreamStorageManager - Single impl, supports either local filesystem or SRB
- Eclipse - BlobStore- Single impl, maps UUIDs to files on disk
- Google Datastore API - Google's "cloud" underneath, REST-ish, container semantics, transactions!...but quite limited today (1MB limit on content, 1000 limit on local files...etc)
- JAF DataSource - Not a Store API; just models a file with a MIME type
- JSR170/283 - JCR; has versioning and container semantics.
- Jakarta Slide ContentStore - Defunct; has versioning semantics
- Jargon API for SRB and iRODS - Models files and directories, with metadata
- JSPWiki FileSystemProvider - Fedora Repository Impl available; API has versioning semantics.
- MediaWiki FileStore - PHP source
- MogileFS - Obtaining
- GlusterFS- a clustered file-system capable of scaling to several peta-bytes
- NFS - Directory semantics.
- OKI Repository OSID - Has container semantics (Assets may contain other assets)
- Plone Large File Management - Several options, but no common interfacealternatives to ZODB. Appear to be explicitly configured as part of a "content type". For example, see FileSystemStorage
- Sakai Content Hosting Service - Collection semantics. JCR impl
- Skwish - Transactions and reads/writes over a collection of "segments" on a filesystem. See Motivation
- SMB/CIFS - Directory semantics.MediaWiki FileStore - PHP source
- Storage Resource Manager - Used by LHC; does not provide direct data access, only references.