Below are the notes I used for the Akubra update in today's architecture meeting.
What's Akubra About?
...
- A standard Java interface for reading/writing files
...
- , but at a different level of abstraction than a
...
- filesystem
...
- Transactional by design (but implementations may ignore transaction semantics)
...
- Exploring web-based exposure
...
- From the Akubra wiki: Requirements and Goals
Note: The Akubra wiki is hosted at http://topazproject.org/akubra
Anyone is welcome to sign up to the dev and user general mailing lists.
Filesystem vs. BlobStore
A normal filesystemCommon filesystems: - has directories
- allows partial reads/
- Have directories
- Can provide system metadata about files (e.g. size, modified date)
- Allow partial reads and writes of files
An Akubra BlobStore: - has
- Has a collection of URI-addressable bitstreams (no "directories")
...
- Only provides the size of each file -- is not concerned with other system metadata (yet?).
- May allow partial reads (InputStreams can skip()...)
...
- Does not allow partial writes
Java API
This is in flux. We are currently testing the design with a simple filesystem implementation.
Blob (A finite, readable bitstream)
BlobStore (For getting connections)
BlobStoreConnection (For CRUD operations)
Transactions Transactions
Level of support varies per-implementation (some can "fake it")
Why: To execute a mixed set of CRUD operations of several files as one atomic unit of work.
Observation: We can build a transactional blob store on top of a non-transactional one
...with the help of a DB.
...
Higher-level BlobStore TBD: -
- Uses FSBlobStore to persist data
...
- Uses database to support transactions (via id mapping)
Other possible storage Plug-Ins:
...
- S3 (anything based on current LLStore should be easy to port over)
...
- ZFS (already transactional, does not need layering)
...
- Centera (content-addressible...ids not available till content is written)
...
- Sam/QFS (hierarchical storage implies graceful handling of delays...not a use case we've factored in yet)
Web-based exposure?
...
- Opens up use of akubra-java impls to other (remotely-running) programs
...
- Allows an akubra impl that's a client to remote akubra instance
...
- Lots of interesting possibilities!
...