Transactions scope: what do we need to support?=20
Individual Resources
Groups of resources
Containment hierarchies
A Fedora transaction may span multiple HTTP Requests. As=
suming that we do not want to commit anything to OCFL until the Fedora Tran=
saction is committed, how do we maintain the state of open OCFL Sessions&nb=
sp;
across requests?
across instance reboots?
across multiple horizontally scaled instances?
How also do we ensure that two requests using the same transac=
tion ID against the same OCFL object do not stomp on each other
Globally accessible state (for horizontal scalability)
Caching/indexing strategy =20
What caches and indexes do we need(ie in what layer(s)?)=20
OCFL client
Persistence Implementation (OCFL)
Kernel Implementation
Physical location of the cache (assuming we want to plan for horizontal=
scalability support)=20
Results were promising using Pete=
r Winckles 's data from UW-Madison
Question: should we consider using Peter Winckles 's java client instead / in addition=
to?
It would be good to fix the travis.yml file to install Go and the ocfl c=
lient.
Meeting part II
Transactions
Action on transactions? Seems like we don't want to preclude arbit=
rary resources in transactions
Also may need to look at the transactions API. The existing transa=
ction API isn't part of the Fedora spec. There is a draft of a new AP=
I, but it has no standing
Question: How much do we have to=
plan about distributed/multiple Fedora instances?
To what extent do we have to plan for this pathway?
Ben Cail: Not interested in multiple Fedora instances. Don't=
anticipate needing to scale, don't want to worry about concurrency. =
Would look at it if it came for free
Maybe that's a question that should go out to the community?
Don't want to get into transaction management.
State/locking
How much can we guarantee?
If we do a transaction involving bits and pieces of multiple OCFL object=
s, it is even possible to roll that back without direct support for these s=
orts of transactions in the OCFL client? Probably not.
Strawman: What if an OCFL client supported such transactions like =
this? How could that be implemented? Is it even desirable?
Write file content to staging places as usual=20
If a failure/rollback happens here, we just have un-referenced staged c=
ontent we can garbage collect later (GC)
Maintain a database table that authoritatively OCFL metadata (that whic=
h gets written to inventory files)
Upon "commit", copy files to the right location in OCFL=20
If this fails, then these copied files can be GC'd later. They ar=
en't referenced by any inventory files, so as far as other OCFL clients are=
concerned, the incompletely-copied files are invisible. That being s=
aid, the OCFL objects they've been copied to are technically invalid until =
these files are removed.
Commit (or roll back) that db table. Report transaction success i=
f that succeeds
Asynchronously, start writing the inventory files to OCFL that referenc=
es the copied content.=20
If there's a failure here, that's OK. The authoritative DB still =
has the information in it to re-try until all the inventory files on the FS=
agree with the db.
TODO: Aaron Birklandflesh th=
is out. Maybe bounce it by the OCFL community. Is an OCFL clien=
t that behaves that way "proper"?
Actions
Danny Bernstein will reach out to G=
reg about DRASTIC test results
Aaron Birkland to look explore notion =
of OCFL client with database as authoritative metadata source + asynchronou=
s writing of the inventory.json file
Peter Eichman and maybe =
Ben Pennell to make recommendations=
re transaction side car specification.