About This Page
The purpose of this page is to inform the design and development of Akubra by providing an analysis of existing Blob Storage APIs and implementations.
Here we define Pluggable Blob Storage API as a programming interface with the following characteristics:
A Pluggable Blob Storage API MUST:
- Be primarily concerned with providing read/write access to named files.
- Be designed with back-end pluggability in mind.
A Pluggable Blob Storage API MAY:
- Provide read/write access to metadata about stored files.
- Provide a way to list or query stored files.
- Provide transactional capabilities.
A Pluggable Blob Storage API MUST NOT:
- Be concerned with container (directory/folder/collection) semantics.
- Be concerned with versioning semantics (e.g. revision history).
APIs and Implementations
API |
Language |
Target Application |
Known Implementations |
---|---|---|---|
Java |
General-Purpose |
AKU-FS: FSBlobStore |
|
Java |
Alfresco CMS |
ALF-FS: FileContentStore |
|
Java |
DSpace 2 (Unreleased) |
BIT-FED: FedoraBitStore |
|
Python |
Any Using Django |
||
Java |
Daisy CMS |
||
COM |
MS SQL Server |
||
Java |
Apache JackRabbit |
||
Java |
Apache JackRabbit |
||
Java |
Fedora Repository (2.x to 3.1) |
LLS-FS: GenericFileSystem |
|
Java |
Topaz OTM |
||
Java / C |
General-Purpose |
Capability Matrix
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
---|---|---|---|---|---|---|---|---|---|
AKU-API |
URI |
User or Store |
Yes |
Yes |
Size |
? |
All |
Optional, JTA |
|
AKU-FS |
file:/// URI |
Store |
Yes |
Yes |
Size |
None |
All |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
ALF-API |
URL |
User or Store |
Yes |
Yes |
Size |
None |
All |
Optional, JTA |
|
ALF-FS |
store:// URL |
User or Store |
Yes |
Yes |
Size |
None |
All |
Optional, JTA |
|
ALF-HTT |
http:// URL |
N/A |
No |
No |
Size |
None |
None |
Optional, JTA |
|
ALF-REP |
URL |
User or Store |
Yes |
Yes |
Size |
None |
All |
Optional, JTA |
|
ALF-ROU |
URL |
User or Store |
Yes |
Yes |
Size |
None |
All |
Optional, JTA |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
BIT-API |
Unicode String |
User or Store |
Yes |
Yes |
Any (Map) |
None |
None |
No |
|
BIT-FED |
(NCNameChar)* |
User or Store |
Yes |
No |
Size |
None |
None |
No |
|
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="8d806f2a-b51b-41ed-ad50-f92559375dfd"><ac:plain-text-body><![CDATA[ |
BIT-FS |
([~cwilper:0-9]){38,40} |
User or Store |
Yes |
Yes |
Size |
None |
None |
No |
BIT-HON |
? |
? |
? |
? |
? |
None |
None |
No |
|
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="241c01a2-bbe1-4a79-ba74-b7f25681ffa8"><ac:plain-text-body><![CDATA[ |
BIT-S3 |
([~cwilper:0-9]){38,40} |
User or Store |
Yes |
Yes |
Size |
None |
None |
No |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="eefa182c-5cc8-4bc4-a8c6-7712602d3f96"><ac:plain-text-body><![CDATA[ |
BIT-SRB |
([~cwilper:0-9]){38,40} |
User or Store |
Yes |
Yes |
Size |
None |
None |
No |
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
DJA-API |
Unicode String |
User |
Yes |
Yes |
Size |
None |
None |
No |
|
DJA-FS |
Unicode String |
User |
Yes |
Yes |
Size |
None |
None |
No |
|
DJA-S3 |
Unicode String |
User |
Yes |
Yes |
Size |
None |
None |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
DSY-API |
Unicode String |
Store |
Yes |
No |
None |
None |
None |
No |
|
DSY-FS |
Unicode String |
Store |
Yes |
No |
None |
None |
None |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="efd6c137-5c02-49cc-a840-35a982153bde"><ac:plain-text-body><![CDATA[ |
EBS-API |
byte[] |
Store |
Yes |
No |
Size |
None |
None |
No |
EBS-FS |
GUID or String |
Store |
Yes |
No |
Size |
None |
None |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
JKB-API |
Unicode String |
Store |
Yes |
Yes |
None |
None |
None |
No |
|
JKB-DB |
propertyId + index |
Store |
Yes |
Yes |
None |
None |
None |
No |
|
JKB-FS |
Filesystem Path |
Store |
Yes |
Yes |
None |
None |
None |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
JKD-API |
Hex String |
Store |
Yes |
No |
Size |
None |
All |
No |
|
JKD-DB |
SHA-1 Digest of Content |
Store |
Yes |
No |
Size |
None |
All |
No |
|
JKD-FS |
SHA-1 Digest of Content |
Store |
Yes |
No |
Size |
None |
All |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
LLS-API |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-FS |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-HON |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-IRD |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-S3 |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-SRB1 |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
LLS-SRB2 |
pid or pid + dsId |
User |
Yes |
Yes |
None |
None |
None |
No |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
OTM-API |
Unicode String |
User |
Yes |
No |
None |
None |
None |
Optional, JTA |
|
OTM-FED |
Unicode String |
User |
Yes |
No |
None |
None |
None |
Best-effort, JTA |
|
OTM-FS |
Unicode String |
User |
Yes |
No |
None |
None |
None |
Yes, JTA |
|
API/Impl |
ID Type |
ID Gen |
Write? |
Rewrite? |
Read-Only MD |
Writable MD |
Listing Options |
Transactions |
|
XAM-API |
XUID |
Store |
Yes |
No 1 |
Last-Modified |
Several Predefined |
All |
No |
|
XAM-CEN |
XUID |
Store |
Yes |
No |
Last-Modified |
Several Predefined |
All |
No |
|
XAM-HON |
XUID |
Store |
Yes |
No |
Last-Modified |
Several Predefined |
All |
No |
|
XAM-IAP |
XUID |
Store |
Yes |
No |
Last-Modified |
Several Predefined |
All |
No |
1 XAM is designed with a fixed-content assumption – data cannot be modified without changing its XUID.
2 In XAM, all metadata fields are either binding or non-binding. Binding metadata cannot change while keeping the same XUID for an XSet, while non-binding metadata can.
Other APIs/Implementations of Interest
- DSpace 1 BitstreamStorageManager - Single impl, supports either local filesystem or SRB
- Eclipse - BlobStore- Single impl, maps UUIDs to files on disk
- JAF DataSource - Not a Store API; just models a file with a MIME type
- Jakarta Slide ContentStore - Defunct; has versioning semantics
- Jargon API for SRB and iRODS - Models files and directories, with metadata
- JSPWiki FileSystemProvider - Fedora Repository Impl available; API has versioning semantics.
- OKI Repository OSID - Has container semantics (Assets may contain other assets)
- Plone Large File Management - Several options, but no common interface
- MediaWiki FileStore - PHP source