You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

Call in details:

Sprinters

Objectives

The objectives of sprint-04 are to:

  1. Test compatibility of exported Bags with:
    1. APTrust
    2. Perseids
    3. Archivematica (question)
  2. Test compatibility of import/export round-tripping between community and alternate Fedora implementations
  3. Import/export of versions
    1. Note: Want to keep upcoming Memento API alignment
  4. Lossless round-tripping (particularly dates)
    1. other round-tripping issues, e.g.:  FCREPO-2429 - Semantic differences between exports Open
  5. Resolve import bugs that were reported from v0.1 testing
  6. Verification tool
    1. Unicode-escaping when resources are exported leads to mismatch when those resources are re-imported
    2. Users are confused by the tool output
    3. Lower priority:
      1. Gracefully handling server-managed triples and timestamps that can't be updated
      2. Verifying Fedora-to-Fedora and disk-to-disk
      3. Anything special required for Bags?
  7. Lower-priority
    1. Importing into non-empty containers
      1. There may be several facets to this: importing into a container that already exists (other children of the same resource already exist) seems very useful, but syncing with an existing resource may be much more complex.  We should support the easy things, and be careful when tackling the more complex scenarios to maintain a good scope with stakeholder input.
    2. Maintain links in Bags with links to resources outside of Bags
      1. Revisit whether this is still an issue given the current Bag export/import functionality
    3. Incremental diffs of Bags, relating to sending updated Bags to preservation services, not re-transmitting large files that haven't changed, and not overwriting good content with content that's been corrupted, etc.
      1. Interesting functionality once the other functionality is solid

Resources

  1. Design - Import - Export
  2. Applications / tools for using and testing the import export
    1. Fedora 4.7.2 one-click (jetty-console jar : https://github.com/fcrepo4/fcrepo4/releases/tag/fcrepo-4.7.2
    2. Fedora 0.1.0 Import / Export utility : https://github.com/fcrepo4-labs/fcrepo-import-export/releases/tag/fcrepo-import-export-0.1.0
    3. Fedora 0.0.3 Import / Export verification utility : https://github.com/fcrepo4-labs/fcrepo-import-export-verify/releases/tag/0.0.3
    4. Steps for using the above

      # Download the utility:
      https://github.com/fcrepo4-labs/fcrepo-import-export/releases

      # Have a running Fedora
      If you do not have a Fedora to test with, you may want to start up the "one-click run":
      https://wiki.duraspace.org/display/FEDORA4x/Quick+Start#QuickStart-One-ClickRun

      # Have data in your Fedora
      If you do not have data in your Fedora, you will likely want to load some of your own or one of the existing example datasets:
      https://github.com/fcrepo4-labs/fcrepo-sample-dataset/tree/master/additional_datasets

      Although there are minimal instructions in the readme linked above, if you run into issues loading any of these datasets, I would be happy to help... the possibility of issues arising is not inconceivable.

      # Export your data as Bagit bags
      Here are some minimal instructions:
      https://github.com/fcrepo4-labs/fcrepo-import-export/blob/master/README.md#running-the-importexport-utility-with-a-bagit-support

      To export APTrust-style bags, use the ''--bag-profile aptrust" argument when running the utility.

  3. Import / Export utility tickets

    type key summary assignee reporter priority status resolution created updated due

    Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  4. Import / Export verification tickets

    type key summary assignee reporter priority status resolution created updated due

    Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Meetings

Meeting 01 - 2017-05-15

  • 9am ET

Attendees

Agenda

  1. Introductions
  2. Logistics
    1. Virtual daily stand-ups?
    2. IRC? 
    3. Conference calls?
    4. Email check-in?
    5. Sprint retrospective?
  3. Sprint Priorities
  4. Assign work

Minutes

  1. Kieran will be out the second week of the sprint
  2. Next meetings
    1. Wed 5/17 @10am ET
    2. Tues 5/23 @3pm ET
    3. Fri 5/19 @11am ET
  3. Dev teams
    1. Esmé and Longshou
    2. Mike and Ben
    3. Josh, Youn and Danny
  4. Testing / Docs teams
    1. Youn / Jim / Kieran / Bridget - detailed testing
  5. New tickets
    1. Import / Export between Fedora and Cavendish - Esmé
    2. Production roundtripping
      1. Duke - Jim
      2. Princeton - Esmé
      3. UMd - Josh
    3. Creation of small dataset for verification testing - Esmé
    4. Plan how versioning will be in line with Fedora specification
  6. Existing tickets
    1. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - resolve with documentation and CND

    2. Longshou / Esmé
      1. Unable to locate Jira server for this macro. It may be due to Application Link configuration.
      2. ...
    3. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - Josh
    4. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - Danny
    5. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - Josh
    6. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - Danny to verify
    7. Unable to locate Jira server for this macro. It may be due to Application Link configuration. - Youn


Standups

IRC Template

[Import/Export Standup]
Finished yesterday: 
  {ticket titles and associated JIRA links}
  {and/or brief textual description}
Working on today:
  {ticket titles and associated JIRA links}
  {and/or brief textual description}
Blockers:
  {brief textual description}

 

<escowles> [Import/Export standup]
<escowles> Finished yesterday:
<escowles> * Adding option to overwrite tombstones on import https://jira.duraspace.org/browse/FCREPO-2202
<escowles> * Suppressing duplicate RDF types on re-import https://jira.duraspace.org/browse/FCREPO-2429
<escowles> Today:
<escowles> * Responding to code review comments
<escowles> * Support changing repository baseURL https://jira.duraspace.org/browse/FCREPO-2335
<escowles> Blockers:
<escowles> * None

<awoods> [Import/Export Standup]
<awoods> Finished yesterday:
<awoods> - Created and assigned several tickets based on Sprint Kickoff meeting: https://wiki.duraspace.org/display/FF/2017-05+Import+-+Export+Sprint+04+Meetings
<awoods> - Reviewed Adding option to overwrite tombstones on import https://jira.duraspace.org/browse/FCREPO-2202
<awoods> Working on today:
<awoods> - Reviewing any tickets that are completed
<awoods> - Working "import tool looks for sha1 manifest file with capitalized filename" :
<awoods> https://jira.duraspace.org/browse/FCREPO-2443
<awoods> Blockers:
<awoods> - None

<youn> [Import/Export Standup]
<youn> Finished yesterday:
<youn> - Ran export and import using one-click jar and 10KIndirectContainers
<youn> - Attempted to run verify.py
<youn> Working on today:
<youn> - Run verify.py
<youn> - FCREPO-2369: verify.py fails on unicode-escaped characters; attempt to replicate error
<youn> - FCREPO-2438: document use of CND for namespaces; attempt to apply fix
<youn> - Will be offline in the afternoon for meetings, etc.
<youn> Blockers:
<youn> - urllib module in verify.py?

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa> Finished yesterday:
<mikeAtUVa> Spoke breifly with benpennell to strategize
<mikeAtUVa> Working on today:
<mikeAtUVa> https://jira.duraspace.org/browse/FCREPO-2432 adding support to update certain server-managed triples
<mikeAtUVa> - replace code that simplifies PUT interactions to use existing "handling-lenient, received-minimal" pattern
<mikeAtUVa> - attempt to implement intuitive date updates adding children with provided creation dates
<mikeAtUVa> - add test cases for binaries and ensure that they work
<mikeAtUVa> - address test failure on travis (didn't appear locally) : https://travis-ci.org/fcrepo4/fcrepo4/builds/232586336?utm_source=github_status&utm_medium=notification
<mikeAtUVa> - determine the interaction of forced modification dates with etags and their implications on caching and if-modified headers
<mikeAtUVa> Blockers:
<mikeAtUVa> - none

<benpennell> Finished yesterday:
<benpennell> * environment setup
<benpennell> * read through Mike's work on https://jira.duraspace.org/browse/FCREPO-2432
<benpennell> Working on today:
<benpennell> * familiarize myself with import/export codebase.
<benpennell> * familiarize myself with current and new versioning specifications, the challenges involved.
<benpennell> * strategize about how to represent versions
<benpennell> * work towards generating some tickets with Mike.
<benpennell> Blockers:
<benpennell> * is there previous work/discussion about exporting versions to reference?


<westgard> [Import/Export Standup]
<westgard> Finished yesterday:
<westgard> re-familiarized myself with code and tickets
<westgard> consulted with dbernstein regarding logging behavior ticket (FCREPO-2368)
<westgard> identified test data that demonstrates bug in unicode escaping (FCREPO-2369)
<westgard> Working on today:
<westgard> FCREPO-2303 (verify triples)
<westgard> FCREPO-2329 (handle missing resources during verification)
<westgard> Blockers:
<westgard> More feedback from stakeholders regarding desired logging behavior & command line options would be helpful (relates to FCREPO-2368)

<coblej>
Finished yesterday:
  Installed applications / tools for import export testing on workstation.
Working on today:
Verifying installation of applications / tools.
Logging documentation for import/export: https://jira.duraspace.org/browse/FCREPO-2425
Blockers:
Cannot load “main” sample datasets into one-click run Fedora 4.7.2.  “mvn -Dfcrepo.url=http://localhost:8080/rest/ exec:java” command results in “ERROR 08:29:39.119 (FedoraInvalidNamespaceExceptionMapper) NamespaceExceptionMapper caught an exception: Prefix fcr has not been registered” for each dataset.  (Loading the “additional datasets” via fcr:restore seems to work fine.)

<lsitu> [Import/Export Standup]
<lsitu> Finished yesterday:
<lsitu> Re-import of hierarchy: https://jira.duraspace.org/browse/FCREPO-2428
<lsitu> Working on today:
<lsitu> Add support for round-tripping with binaries excluded: https://jira.duraspace.org/browse/FCREPO-2426
<lsitu> Blockers:
<lsitu> N/A.

<dbernstein> [Import/Export Standup]
<dbernstein> Finished yesterday:
<dbernstein> https://jira.duraspace.org/browse/FCREPO-2448
<dbernstein> * Added setup.py to verify tool and setup up support for adding unit tests
<dbernstein> * Did some minor refactoring of command line parser using ‘click’ module
<dbernstein> * Add support for installing verify (now fcrepo-verify) on the path.
<dbernstein> * Pull request is in.
<dbernstein>
<dbernstein> Working on today:
<dbernstein>
<dbernstein> * https://jira.duraspace.org/browse/FCREPO-2368
<dbernstein> (provide full DEBUG-level logging for import-export verification tool)
<dbernstein> Blockers:
<dbernstein> None.

<bridgetalmas> [Import/Export Standup] Working on today:   Verify Perseids BagIt Profile  https://jira.duraspace.org/browse/FCREPO-2440 Blockers:   Mapping to LDP model   Import of tag manifest contents (https://jira.duraspace.org/browse/FCREPO-2444)

<youn> [Import/Export Standup]
<youn> Finished yesterday:
<youn> - Ran verify.py to verify export of 10KIndirectContainers; attempted to verify import but stopped when I ran into lots of verification errors
<youn> - Started repository with modified repository.json and namespaces.cnd
<youn> Working on today:
<youn> - https://jira.duraspace.org/browse/FCREPO-2369 - backup and import data set that caused error
<youn> - https://jira.duraspace.org/browse/FCREPO-2438 - finish
<youn> - look into import verification errors
<youn> Blockers:
<youn> - none
<youn> Correction: - https://jira.duraspace.org/browse/FCREPO-2369 - restore and export data set that caused error

<coblej> [Import/Export Standup]
<coblej> Finished yesterday:
<coblej> Verified installation of applications / tools.
<coblej> Working on today:
<coblej> Logging documentation for import/export: https://jira.duraspace.org/browse/FCREPO-2425
<coblej> Testing with subset of production-like Duke data: https://jira.duraspace.org/browse/FCREPO-2450
<coblej> Blockers:
<coblej> None

<escowles> [Import/Export Standup]
<escowles> Yesterday:
<escowles> * Updated PR for overwriting tombstones: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/76
<escowles> * Worked on support for changing repository baseURL https://jira.duraspace.org/browse/FCREPO-2335
<escowles> Today:
<escowles> * Continue working on changing repository baseURL
<escowles> * Meetings
<escowles> Blockers:
<escowles> * None

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa> Finished yesterday:
<mikeAtUVa> - https://jira.duraspace.org/browse/FCREPO-2432
<mikeAtUVa> - rolled back changes (in my work) that simplified PUT semantics
<mikeAtUVa> - identified additional work necessary to use existing "minimal" triples requests
<mikeAtUVa> - learned some more about how binary properties are stored, devised strategy for updating previously server-managed properties for binaries
<mikeAtUVa> Working on today:
<mikeAtUVa> - Hope to finish up https://jira.duraspace.org/browse/FCREPO-2432
<mikeAtUVa> Blockers:
<mikeAtUVa> - None

<awoods> [Import/Export Standup]
<awoods> Finished yesterday:
<awoods> - Completed: "import tool looks for sha1 manifest file with capitalized filename" : https://jira.duraspace.org/browse/FCREPO-2443
<awoods> - Reviewed: "Add ability to delete or overwrite tombstone on import" : https://jira.duraspace.org/browse/FCREPO-2202
<awoods> - Reviewed / Merged: "Semantic differences between exports" : https://jira.duraspace.org/browse/FCREPO-2429
<awoods> Working on today:
<awoods> - Reviewing any tickets that are completed
<awoods> Blockers:
<awoods> - None

  • No labels