Versions Compared

Old Version 1

changes.mady.by.user Danny Bernstein

Saved on

compared with

New Version Current

changes.mady.by.user Carrick Rogers

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Attendee

Agenda


  1. Progress Report on  Fcrepo-Cloud-Migrator Tool
  2. Message emission in for inbound link targets.
    1. Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyFCREPO-2803
    2.  
      Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyFCREPO-2804
       
  3. Review Spec committee's direction on Fedora API Specification github issue (one-step fixity check)
    1. Also what about dangling question about storing checksums as a server managed triple? 
  4. Sprint doodle poll results:  https://doodle.com/poll/2vgw3huvr8dugeuq:  emerging consensus
    1. Sept 10-24
    2. Oct 1-14 
  5. OR:  
    1. Dinner Meetup? 
    2. T-Shirts
    3. Intro Workshop
    4. Talks you're giving,  talks you're excited about?
  6. Feedback on /fcr:fixity  endpoint
    1. Fedora API Specification github issue
  7. Getting to 5.0 
    1. Review of work that happened this week: 
      1. PRs merged for 
        1. Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-2792

        2. Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-2795

      2. Remaining external content (low-hanging: ie volunteers?) issues:

        External content (proxy, copy, and redirect)  - PR merged, there are a few issues relating to it that are low hanging fruit. 

        Expand

        Jira
        serverDuraSpace JIRA
        columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
        maximumIssues20
        jqlQuerykey in (FCREPO-2776,FCREPO-2801,FCREPO-2752,FCREPO-2802,FCREPO-2796,FCREPO-2797)
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


      1. 5.x Documentation Effort
        1. 5.x Documentation Updates matrix
      Another sprint? 
      1. Timeframe/Duration/Availability
    2. Ecosystem tools dependent on the release:progress report
      1. import export tool
      2. camel tool kit  
      3. fcrepo4-vagrant
      4. java client 
      5. api-x 
  8. Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyFCREPO-2604
    : 

    1. Are we comfortable with messages emitted?

  9. Progress Report on Fcrepo-Cloud-Migrator Tool
    1. variants in .ttl between the python export and ruby-rdf
  10. Reviving OAI Support 

Ticket Summaries

    1. Please squash a bug!

      Expand

      Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      maximumIssues20
      jqlQueryfilter=13122
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


    2. Tickets resolved this week:

      Expand

      Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      maximumIssues20
      jqlQueryfilter=13111
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


    3. Tickets created this week:

      Expand

      Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      maximumIssues20
      jqlQueryfilter=13029
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


Minutes

  1. fcr:fixity endpoint
    1. discussed in fcrepo leadership
    2. Andrew: fixity as a test case that highlights need for clear feedback and communication channels between community, tech, leaders, and committers
    3. tech thread and leadership thread: do we want fixity service? yes
      1. open discussions about resources needed to maintain this feature
      2. if it is important for community it should be in the spec
    4. github issue: https://github.com/fcrepo/fcrepo-specification/issues/373
      1. open question about whether there could be an additional header to RFC 3230 that returns a success/failure status
    5. deprecation status?
      1. should it be in the core? seems to be a ciritial preservation feature
      2. Peter: fine with moving this to a header if we can get a status header a la Andrew's GitHub issue
      3. Josh: agrees that this needs to be a core responsibility of Fedora
        1. external content wrinkle
        2. external content is beneficial to large content repos
        3. external content is not covered by current fixity
        4. Josh has been looking at extensions to check fixity for external content
      4. Bethany: copy and proxy will check fixity on new external content implementation; only redirect is left out
        1. could use some further testing
        2. Josh: how does it perform?
    6. Andrew: interested in the header approach
      1. Peter: can we use a link header for this?
    7. Bethany: are the spec writers talking about this?
      1. Andrew: no, this is why github issue has been added
    8. Jared: where do we store the digests? have to persist check
      1. external content is a bigger concern for performance
      2. redirect seems hard to implement
      3. Jared: need to store checksum where it is always available; similar to MIME type
      4. Peter: sounds like a server-managed triple?
        1. Bethany: store as a property on the binary itself?
        2. Jared: if server-managed triple do you return it in your graph?
        3. Andrew: these seem to be issues in general about server-managed triples
      5. Jared: the reference implementation should only implement the spec
        1. adding additional features to reference implementation means the spec is lacking
        2. Josh: API-X as an alternative to core?
        3. Jared: should also be discussion around the github issue
        4. if it is a vital part of Fedora then it should be in the spec; discuss how (MUST/MAY/SHOULD?)
        5. Andrew: agrees essential features should be in spec
      6. fixity service only answers the question of "is this resource what you think it is?"
        1. "self-healing repo" feature is out of scope; current/future fixity service does not address the question of "what next?"
        2. Jared: interested parties should speak up in comments on the github issue
        3. Bethany: we should send a follow-up email on the original thread(s) that highlights the github issue
      7. Doran: seems to be two issues: a) should we have the service? b) how should it be implemented?
  2. getting to 5.0
    1. PR for external content went in this week
      1. handful of issues to check
      2. any other 5.0 developments last week? no
    2. documentation updates
      1. ongoing effort
      2. Andrew: note the red X's indicating pages that need work; reviewer should add notes about what kind of changes are suggested
      3. Josh: can contribute to documentation effort
        1. are there instructions about what should be done for updates? procedure?
        2. Bethany: just go in and edit the 5.x versions of pages
        3. if you work on a page work on the whole page, or note where you stop
      4. Kevin: reviewer status?
        1. Peter: use the star
        2. Bethany: do we need to track who reviewed?
        3. Kevin: add reviewer name in the "Who?" column
    3. another sprint
      1. timeframe: July? maybe 2 sprints
      2. Andrew: did a Doodle go out
        1. no one seems to have seen a Doodle
        2. action: send out a Doodle to get a timeframe (Andrew)
      3. Andrew: 2 sprints sound good
        1. earliest/latest dates? July-October
  3. messages emitted
    1. what do we need to do to move FCREPO-2604 along?
    2. Andrew: Kevin's point: does a change to a inbound reference warrant a message?
      1. if yes, close ticket as complete
      2. if no, need more work
    3. Josh: inbound reference shouldn't change the state of the target resource
      1. Bethany: agrees
    4. Kevin: implication is just for resources that need to know that inbound references changed
    5. Bethany: can we distinguish between different kinds of changes
      1. Andrew: we don't track individual properties, so might be weird if we track this but not individual properties
    6. Josh: could this be useful for indexing
      1. Peter: does this have enough information to be useful for indexing
    7. Bethany: puts the work on the client?
    8. Jared: can we use different ActivityStream type for these type of change messages?
      1. there are object/link type relationships
      2. makes for easier filtering
      3. Bethany, Peter: agree
      4. Andrew: requires Fedora to know that a change is due to an inbound reference
      5. action: new JIRA ticket to explore nature of changes before messages are emitted (Jared)
  4. fcrepo cloud migrator tool
    1. Carrick: can do an overview next week
    2. has anyone tried ingesting edited Turtle? not yet
    3. Peter: looks like RDF 1.0 to 1.1 conversion issue (explicit to implicit string type)

...

  1. Fcrepo-cloud-migrator update
    • Northwestern need to move to an external s3 based resources and multitenancy.  
    • Ruby script ties into the import export tool
    • Fully tested by the end of June.
    • Export is 60 GB  (binaries, rdf, versions)
    • Tool will be useful to anyone migrating from a Fedora 4 instance with locally managed binaries to a cloud-based Fedora with binaries externally managed in S3. 
    • Should be mentioned in the newsletter.
  2. Issues related Indexing not burdening the system
    • Request is not to send out two events when one item is linked to another item (one per item), 2803 is to figure out where in the code base the event is kicked off and create a different event that could be filtered via listener.  Ideally solution is to recategorize the event so the events are still there if you want the event (for reindexing, etc) but it is not clear if this is a viable solution.  General consensus on continuing to generate the events but make them easier to filter.  2806 stems from discussion from 2604.
  3. Comments from fixity verification meeting but minimal discussion topics.  It seems that the community wants this and the issue is figuring out the best path forward to implement this/API changes involved in this.  
    • Implementation question where the checksums would be stored
    • Potentially add automatic calculation of SHA1 for external files (AWS S3 does MD5 and SHA256 automatically)
    • Upon creation of an external binary, provide the checksum as you know it, should Fedora validate that checksum and then store it?  It might/should work as is via verify checksums.  Does require streaming the content to Fedora
  4. Sprint Doodle Poll Results, poll is ongoing additional names added but consensus seems to remain around those dates
    • Possible August dates, wait for more people to sign up
    • Poll closes at the end of the week: perhaps we should consider extending it a couple more weeks
  5. If you are going to OR contact Danny Bernstein about organizing an OR dinner, no day set yet.
    • Teeshirts will also be available at OR, if you're interested ask Danny on slack about size of the teeshirt
    • Danny is giving a workshop on Fedora (40 to 60 people estimated), support would be appreciated for the hands on section if anyone else wants to come
    • Yinlin Chen will give an OR talk June 6th, link to the talk: Toward Cloud-Native Digital Repositories
    • Doron Shalvi will be at OR and is interested in talking about fixity and cloud migration 
  6. Couple PRs merged over the past week, 
    • Any interest in tackling the low hanging issues regarding external content?  Should be easy to get in and get out on
    • Documentation is moving well, kudos to Kevin Ford for tackling many of the issues
    • Much of the documentation should just be verification, so many of the pages can be reviewed quickly (under 30 mins), others could contribute easily 
    • Someone needs to send out an email to fedora-tech to see if anyone is using the java client without the camel toolkit (can the camel toolkit be updated without having to update the java client)
  7. Questions came up at Fedora camp about OAI support
    • Seems like there would be interest in an OAI plugin or extension 
    • Appears there was one in an early version of 4 (last offered 4.4), now broken.  What is the interest level in bringing that back?
    • Earlier version was tied to features that have since been dropped, such as internal search (note that internal search might come back).  Would need a rewrite to bring back.  If it was easy it would have already been been.
    • Email on the listserv from March with Andrew Woods encouraging the community to revive the feature but no traction
    • Need a user and someone to maintain it, otherwise something would just get written, neglected, and eventually break
  8. ImportExport Tool

Next week's meeting is cancelled due to OR

    •  Write JIRA ticket regarding calculating the SHA1 checksums Carrick Rogers (
      Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyFCREPO-2805
      )
    •  Consider Extending Doodle Poll to later date, it closes this Friday (Danny Bernstein)
    •  Send an email to fedora-tech seeing if anyone is using the java client (people are and importexport tool uses it)
    •  Send email to community seeing who is interested in reviving OAI

    ...