2012-04-02 DfR Technical Meeting

Regular Attendees

• Andrew
• Bill
• Brad
• Chris
• Dan
• Jonathan (Presumed at CNI)

General

• Call In To: Free Conference Call HD - DuraCloud Line

  Please note that we will be using the Free Conference Call HD line for this call. Information about calling into this line is available from the link above.

• - Indicates who will be taking minutes

Discussion Topics

Planning Board
Task Board
DfR Planning and Estimating - 0.1

Actions from last meeting

Last meeting

Last Weeks Actions:

Related Information
DfR Software Next Steps - Jan 2012
2012-01-03 - Architecture Meeting, Temple University

Status

• Jonathan
• Andrew
• Bill
• Brad
• Chris
• Dan
  • Working overview documentation.
  • Work on potential additions to managing the synch tool (adds to tool and UI requirements)(coordinated with Bill, Chris on Box test lessons-learned).
  • Still looking Islandora review

Minutes

• Bill: concerns from DuraCloud mtg that we aren't going to make code cutoff until end of week; testing starts next week, release following Fri/Mon.
  • Dan:  Anything for DfR that won't make it into the DuraCloud release?
  • Areas of concern:  automated bit integrity will be in there; also bug fixes.
• Bill (more notes to follow) - LDCX / Hydra highlights:
  • Lots of interest in DfR at LDCX - retake of arch done on a slide for LDCX and Hydra Partners;  good questions and discussion;  Tom Cramer excited about the sync tooling that can drive OCS.  Discussed possibilities for integrating tech via openstack or S3 APIs with their local content, and reusing the technology for object creation.
  • Discussions with Ben Armintor about Fedora backed up to the cloud; three models:
    • Hold externally (e.g. in cloudsync)
    • Hold replication status in a fedora object
    • Include the replication status in each object in a special data stream (unversioned xml?)
  • Discussions with Richard Green about some content local and some with primary storage on duracloud, all fedora managed.
    • Proposed a 'C' data type - a special version of an external stream which is essentially flagged as being external.  Might be a rels int flag for this as well.
    • Mentioned that this might also address questions of writeable / non writable external streams;  async or sync behaviors; ability or need to chunk for large transfers.
    • N.B.  some of our code is flagging for HTTP chunk-mode, but not clear that it is working correctly.
    • This could be handled by disseminators, or handled at fedora's storage layer?  However, if needs to chunked at storage layer, might need to be chunked to/from the client as well.
  • Anyone using Shib at other than the web tier?  No.  At the lower tiers, no real information on if any credentials or auth is in use inside.
  • At LDCX, about 10 of 30 hands went up for "we're using shib" - but likely duplication  = 1/3 or less are actively using today.
  • Stanford using Argo (locally developed, hydra based) for workflow management and admin console for their workdo workflow.  Not scriptable yet, expected to be soon;  offered to DuraSpace for use in DfR if we want.
    • Is this too Ruby centric?
    • Would it be of general use to the community?
  • For orchestration, is Hull staying with jBPM, what are others using?  (Unknown)
  • Hylandora space on the wiki shows the objects that are created by both systems that shows what needs to be created.
• Preparing to close the DfR 1st iteration:
  • What items are outstanding?
  • DFR-76?  Moving to next iteration?  (Yes).
  • DFR-77 should make it, will be placed in progress.
  • DFR-78 Doc task enhanced by Bill's diagrams, to complete by Friday
  • DFR-20; DFR-5, DFR-4 running behind.  Have a running Islandora generic image; expect to have done by Friday.
  • Islandora trend is to place more in the Fedora objects and less in Drupal
  • (Do we need to formalize our understanding with DGI?  Yes; expect progress in meeting in two weeks.)
  • Security:  Prototyping for Shib IDP and service provider and spring security integration with LDAP have all gone well.  Understand where to place a CAS server if required.  See demo.  Everyone has a username and pw with "pw" after username.  ModShib takes header elements from IDP and passes them to the web layer.
  • Also, Andrew has a standard Duracloud with extensions running locally - can ask shib to provide everything needed to 
  • Wiki pages: htps://wiki.duraspace.org/display/DFR/Shibboleth+IdP+Setup,  https://wiki.duraspace.org/display/DFR/Shibboleth+SP+Setup.
  • Range of good options for bringing CAS server in under Spring Security.  Closing Jira items now.
  • Still need to work out how Islandora works with Shib.
  • Need to have Islandora / Smithsonian system up so that we can work through the issues, especially roles.
  • Need to understand how Islandora / Fedora XACML connection works.
  • DFR-2, DFR-59 can be closed; DFR-60 can be opened and closed based on the work done to date.  https://wiki.duraspace.org/display/DFR/DfR+AMIs
  • DFR-74 will be closed when duracloud issue closes.
• Still open:
  • DFR-23 to be in progress, and try to complete first set by end of week.
  • DFR-8 some small progress, place in progress.
• Next Monday, planning the next iteration;  please start building a page of subjects to discussion during the planning meetings (or add to Jira).
• Andrew:  Do we want something demonstrable for end of first round?  (It will be documentation, Andrew's site, Chris' code; no other major show and tell.  Will prepare a summary of wok to send to execs,  Hope to define something demoable for the end of rev two).
• Bill:  Can we prepare a list of the components that go into DFR and have their statuses at end of 0.1?  E.G. Release Notes for a non-release?