Versions Compared

Old Version 1

changes.mady.by.user Danny Bernstein

Saved on

compared with

New Version Current

changes.mady.by.user Ben Pennell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Danny Bernstein
  2. Jared Whiklo 
  3. Andrew Woods 
  4. Peter Winckles  
  5. David Wilcox  
  6. Ben Cail 
  7. Thomas Bernhart (star)
  8. Daniel Lamb
  9. Ben Pennell (star)
  10. Rosie Le Faive

Agenda

  1. Announcements?

    1. Update on Fedora CI - sonatype and docker
    2. June 1-5 Sprint
  2. May 4-8 Sprint - Wrap up
    1. What's done
    2. Tickets needing Review Updates
        Simple Search - 
      1. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-3298
        Migration tooling
        -3182
      2. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-3193
        - 
      3. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-31823282
      4. WebAC - 
        Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-3294
      5. Testing
      In-Review
      2. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-3296
      3. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-32913299
      4. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-32963302
      5. Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyFCREPO-32983304
    3. Demo
  3. Database support
    1. Moving Fedora Id → OCFL Id index to database - 
      Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyFCREPO-
      3297
  4. Alignment on "Fedora OCFL Storage" proposal
    1. 3309
    2. How to deal with SQL Differences (H2, mysql, postgresql)
  6. Fedora 6 Configuration
    1. https://docs.google.com/document/d/1EuB7QThWP2HhnAqztmUyJgbaxciBa0ySI1ShsciayRE/edit?usp=sharing
    2. Goal: review settings and determine what we want the user experience to be
    3. Reference: Best Practices - Fedora Configuration
  7. Open questions: 
    1.  JMS: Remove Fedora-related JMS messaging headers? 2018-04-26 - Fedora Tech Meeting
  8. Containment Index
    1. How to handle SQL differences (H2, MySQL, PostgreSQL) - pluggable connectors?
  9. Change versioning default to opt-out of auto-versioning?
  11. Community-raised issues
    1. Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyFCREPO-3313
    <Add your topic here>


  12. Expand
    titleTickets resolved this week...

    Jira
    serverDuraSpace JIRA
    jqlQueryfilter=13111
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


...

  1. In Review

    Expand

    Jira
    serverDuraSpace JIRA
    jqlQueryfilter=13100
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  2. Please squash a bug!

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13122
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  3. Tickets resolved this week:

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13111
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  4. Tickets created this week:

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13029
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


Notes

  1. Announcements

David Wilcox : Summarizing current work regarding community interactions:

  • Online Fedora 
  • Building training materials for Fedora 6 → who would/can contribute?
  • One online user group meetings/trainings in US
  • probably another in European timezone
  • Announcements will follow
  • Goal keep up community interactions
  • fcrepo-terraform project –> can it be made public?

Sprint updates:

Danny Bernstein : Summarizing progress of current sprint:

...

...

...

  • API test suite reactivated by Andrew Woods
  • a lot of tests are already passing

...

    1. Docker - Thomas and Danny finishing fcrepo-docker, scripts for building it.
      1. when fcrepo deployment is complete, it will deploy to sonatype, clone fcrepo-docker and run it
      2. Didn't use docker actions because they are triggered by commits rather than complete deployments
  2. May Sprint wrap-up
    1. Simple search project started, started code and specification
      1. Documented, still need to transfer it to the wiki
      2. Code started, initial endpoints in a PR
    2. Migration utils
      1. PR to produce fedora 6 objects or ocfl without fedora aspects
        1. Peter ran Fedora 6 on some migrated Fedora 3 content
        2. There is a University of Wisconsin test data set which will be made available
    3. WebAC
      1. Done, pending feedback
      2. Difficult to get the base uri in while still using Shiro due to the type of filter class, but there is a similar
      3. Could use some caching, it would reload webacs multiple times per request
      4. Some followups:
        1. Not getting all rdf:types back to match on accessToClass -
          Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-3279

        2. Content-type with charset causes it to be a binary -
          Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-

...

        1. 3312

      2. Some tests ignored pending other tickets:
        1. PATCH requests are not parsed with the HttpRdfService so internal identifiers are not stored
          Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-

...

        1. 3311
        2. DELETE DATA sparql-updates appear to succeed but don’t happen
          Jira
          serverDuraSpace JIRA
          serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
          keyFCREPO-3310
    2. Testing
      1. Andrew covered transactions
      2. Published results from running API test suite
        1. Requires a PR to update some dependencies due to java 11
    3. Tickets in review
      1. 3302 - Autoversioning as default for fcrepo6, whereas before it defaulted to off
      2. 3304 - Trimmed out some parts of the PR description template
    4. Demo
      1. Migration utils changes
  2. Database support
    1. Need to support mysql and postgres for production indexes. For testing, using h2.
      1. This will require differences in SQL operations per database
      2. OCFL client - detects what datasource it is to determine what database it is. Fedora then decides how to configure itself
        1. Determine if we need separate classes per index implementation
      3. Will start by targeting mysql and postgres
    2. Still need to move mapping of ocfl to fedora ids into database, it is still in memory. Danny will most likely work on this.
    3. Should the default h2 implementation write to disk?
  3. Configuration
    1. Currently two ways to configure
      1. fcrepo-config.xml - spring context file, with system property overrides. Sometimes modified, sometimes system properties used
      2. System properties - sometimes used in code
    2. Previously there was repository.json, including db connection info, storage path, etc
    3. Proposal
      1. On startup, read property files, then use switches from these properties to determine what to load. Don't require spring context modification.
      2. Can provide the properties either via this file or via commandline properties
      3. Intent - improve usability of configuration

Alignment on "Fedora OCFL Storage"

Peter Winckles : summarizing proposal from https://pad.disroot.org/p/874NsxcjLLAAwgIRg2kh

  • OCFL objects should be transparent → files on disk match expectations of users what should be in that object
  • handle conflicting filenames in OCFL storage, 3 approaches:
    • adding a reserved prefix/suffix → users would be unable to create resources with that prefix/suffix
      • certain files created by fedora will have that prefix/suffix
    • store system generated files in separate directory
      • not outlined in document
      • downside: more directories
      • resource is never in the OCFL object root directory → might be unexpected by users
    • no expectation where a ressource is stored
      • Fedora would have to check the resource headers to find conflicts
      • conflicts are resolved by adding a mapping to the resource headers (if a file needs to be stored under a different name)
      • most of the time, files are exactly where they should be, unless there is a collision
  • approaches should be discussed in separate meeting
  • main question: transparency of ocfl vs. easier to handle implementation
  • Ben Cail favors solution with separate directories
  • discussion related to handling Vanilla OCFL –> one approach here is to treat all files Vanilla OCFL objects as binaries
  • proposal by Andrew Woods : don't replicate the fedora id into the filename, but use fixed names

Remove Fedora-related JMS messaging headers

Not discussed this week.

Containment Index

Not discussed this week.

Change versioning default to opt-out of auto-versioning?

Not discussed this week.

Actions

  •  Danny Bernstein to ensure that a JIRA exists for scenario of transaction does not succeed completely, but rollback does succeed (see: 187172642 2020-05-14 - Fedora Tech Meeting)
  •  Peter Winckles to create a JIRA to investigate support for optimistic locking and mutable head (see: 187172642 2020-05-14 - Fedora Tech Meeting)
  •  Who:  Clarify in documentation that multiple simultaneous writes to OCFL are not supported
  •   Who:  After team has a chance to comment, send Ghost Node idea to general community for feedback
  •   Who: Decide who works on what in the next sprint
  •  Thomas Bernhart Update some of documentation pages for Fedora 6 (with findings from building the docker container)
  •  Danny Bernstein Create ticket related to caching webacs
  •  Danny Bernstein Create ticket to determine how we want to handle differences between index implementations