Call in details:

Sprinters

Developers

Testing and Validation

Documentation

Meetings

Meeting 01 - August 29, 2016

Agenda

  1. Introductions
  2. Logistics
    1. Virtual daily stand-ups?
    2. IRC? 
    3. Conference calls?
    4. Email check-in?
    5. Sprint retrospective?
  3. Phase 1 priorities
    1. Support transacting in RDF
    2. Support allowing the option to include Binaries
    3. Support references from exported resources to other exported resources
    4. Support import into a non-existing Fedora container
    5. Support export of resource and its "members" based on the ldp:contains predicate
    6. The URIs of the round-tripped resources must be the same as the original URIs
  4. Assign work

Minutes

  1. Introductions
  2. Logistics
    1. Virtual daily stand-ups on IRC by 10 am ET
      1. What you did yesterday
      2. What you plan on doing today
      3. Blockers
    2. Conference calls on this line if needed
    3. Sprint retrospective on Sep 9 at 3:30 pm ET
      1. What worked
      2. What didn’t work
      3. Try to wrap up by Thursday
  3. Phase I Priorities
    1. Camel serializer
      1. Supports transacting in RDF and has option to include binaries
      2. Listens for messages on queues; need to run indexer before running export
      3. Basic case: dump to disk and gzip
      4. Need to parse RDF to figure out information required for import, e.g., base url for repository
      5. Provides options for RDF serialization supported by REST API and supplies default serialization
  4. Looking ahead to Phases II and III
    1. Versions
      1. JCR versioning will include any version of parent
      2. For Camel serializer, does Fedora send out a message when it creates a version?
      3. Possible way forward is to consider basic case as export of current version and then to build in capability to export version that is not the current version with additional metadata
      4. Need to consider layout of versions on disk
      5. Fedora API specification is leaning towards making versions first-class resources
    2. Bags
      1. Client tool
      2. Generate from REST API calls to repository or from filesystem?
      3. Only LDP basic containers? Connected graph from root; basic containers if not starting at root
      4. Stakeholder use cases include exporting individual resources, not entire graph, and being able to generate resources that can be imported into other repositories (such as APT) from documented export formats
  5. Design considerations
    1. Camel may make bar too high for users; code may need to be re-implemented
    2. Some discussion of using bash or python for prototyping; final consensus was to use java
    3. Testing plan should define success, describe tests that will be run, and include test data
    4. Integration test suite in java; unit tests included with utilities
    5. Logging
  6. In-flight tickets
    1. FCREPO-2127: Document how the Camel RDF Serializer exports content to disk
    2. FCREPO-2128: Review Camel RDF Serializer implementation
    3. FCREPO-2129: Create import-export github repository in the fcrepo4-labs organization
    4. FCREPO-2130 (close out FCREPO-1990): Create skeletal import client utility
    5. FCREPO-2131: Create skeletal export client utility
    6. FCREPO-2132: Create a test plan
    7. FCREPO-2133: Create sample test dataset
    8. FCREPO-2134: Document sprint resources
    9. FCREPO-2135: Create user documentation
    10. FCREPO-2136: Create Import/Export wiki documentation

 

Meeting 02 - September 2, 2016 
 

Agenda

  1. Open issues
  2. Review ticket process
  3. Squashing commits
  4. Review use of IRC
  5. Helping everyone use jar and load data
    1. We need data with cross-references and external references(question)
  6. Consider refactoring export to have 'ldp:contains' passed in?
  7. IMPORT!
  8. Review tickets
  9. AuthZ

Minutes


Ticket review

  • 2127 justin will add step by step instructions with output to this ticket, to provide more info.
  • 2130 Esmé  will take over ticket, and start on import utility, get done what he can for others to review on Monday.  this will be a basic, skeletal version, working with a small repo, containing a couple of small handbuilt resources.
  • 2132 we reviewed Youn's test plan doc and resolved comments https://docs.google.com/document/d/1WW0dU9LDWvnRPGbzKpIE-ODkGq39md17sKkWVXC0P8U/edit
    Nick and Youn will work on adding this to the wiki
  • 2133 Josh has sample data in sample data repo, almost ready to close ticket.
    Youn has sample data, and is working on a script to put that data into a fedora repo, currently living in a google drive folder.
    Esmé provided feedback on how to modify the existing sample owl file, Justin will help next week with scripting the loading of owl data, if needed.
  • 2134 will leave open for extra comment/input untill next week
  • 2135 leaving open and unassigned until next week
  • 2139 low priority
  • 2143 and 2144 leaving open til next week
  • 2146 discussed, needs to be revisited, once basic import tool exists - some discussion about how to pass in and save config (command line and
  • 2155 Andrew is working on integration tests - run export and import from command line, to test roundtripping.
    Esmé suggested writing the test in the other order - import a fixture, then export it.  Andrew will commit initial driver level tests first, then create a follow on ticket for sub-module level tests


    Some take aways from the ticket review discussion:
    General process for making sample datasets:
    write up a document that provides links to raw data
    write a script to import that into a fedora repo
    do an fcr:backup on that repo
    tar up the output of fcr:backup
    add all the outputs from above steps to fcrepo sampledata github repo

    small test fixtures will be tested from integration tests
    larger sample datasets (like in fcrepo sampledata) will be tested that work with the fcrepo-vagrant as the test environment, and documented in the test plan, which will be added to the wiki.

    we reviewed process for how to use jira:
    -make sure to take ownership of tickets you are working on
    -mark them as in progress, or ready for review, etc as you go

    we reviewed the Pull Request process:
    -put in a pull request
    -others comments
    -based on comments, you make subsequent commits (don't squash)
    -once the pull request is complete, then squash down to one commit before merging to master

    This is not a hard and fast rule, there can be exceptions,
    -sometimes it is better to not squash, or squash to a set of commits, if there is logical separation between commits
    -sometimes the branch has to be rebased, due to other work going into master, and in that case it can make sense to squash when rebasing.

    Esmé provided a link to a good blog post on squashing commits by Jeremy Friesen: http://ndlib.github.io/practices/one-commit-per-pull-request/

    discussed communication
    -using irc, make sure irc client can notify you when your name is used, try to keep irc on so you can participate during the sprint

    discussed how to actually get the code, making sure all of us can build and/or acquire the jar file and run it. Ask in irc if there are questions.

    talked about process of dependency management and use of ldp:contains, Esmé and Andrew will do some refactoring.

Meeting 03 - September 7, 2016 

Agenda

  1. Goal: complete all phase-1 requirements and documentation
    1. Support transacting in RDF
    2. Support allowing the option to include Binaries
    3. Support references from exported resources to other exported resources
    4. Support import into a non-existing Fedora container
    5. Support export of resource and its "members" based on the ldp:contains predicate
    6. The URIs of the round-tripped resources must be the same as the original URIs
  2. Open pull requests
    1. https://github.com/fcrepo4-labs/fcrepo-import-export/pull/17
    2. https://github.com/fcrepo4-labs/fcrepo-import-export/pull/22
  3. Check-in on importer
  4. ...

Minutes

 

  • Sprint goals: Once the current PR that Esmé has is merged (PR-22), we will be done with the crossed off items on the list from #1 above,.
    • Andrew - has a ticket that's in progress where a config file is created upon export and can be used by import and export - the options will be written to a file and on subsequent runs can be passed in and used.
    • Config should/could contain at least the Base URI and directories where stuff is store. Both of those would tell how to map the URIs in exported RDF to files on disk.
    • Bags
      • Bags - next phase export utility will have to generate bags? or can you convert it to a bag? Whatever is making bags will need the config Andrew's working on to know how mapping works. Should exporter export bags or should we have a utility to take an export and convert to a bag? Unclear as to which direction to go in. Postoned talking about bags to look at tickets, since bags are not part of phase one.
    • Import Utility
      • If you try to restore RDF - it will only work if you import into repo at the same url, because no mapping is being done. You might want/expect a new URL if you move to new system. The import will not work in that scenario, as it's currently implemented.
      • Import would need to know the old baseURL and the new baseURL. Would it be helpful instead on the export side of things to doing a stream replace on base url - replace with fedora:info? That way you don't need to know the base URL that was exported from, just put it here. Will this help us out with bags down the line? Unsure.
    • Export Utility
      • Esmé will make a ticket for replacing baseURL with fedora:info in the export.
  • Leave call knowing how we will meeting a-f as well as documentation. Completion of phase 1 is defined by a-f being done, a utility to point people to, and a top level wiki page that describes what we've done and how to use it.
  • Pull Requests:
  • open tickets:
    • FCREPO-2166 & FCREPO-2167 - Josh will finish today.
      • Have dataset with cross object relationships & relationships to external content.
      • There is a python script that's a batch loader specific for this set of content. If someone wants to run the python to load it, contact Josh. Josh created a ticket for the namespace issue he was seeing yesterday.
    • FCREPO-2168 - Josh can also work on this one.
    • FCREPO-2132 - Nick wants feedback. Leave this open throughout all phases of sprints? How to proceed: It needs a section talking about what we are going to do to validate the utilites. One more section with bullets so that someone could, potentially, run the tests themselves. After that closing theissue makes sense.
    • Testing - how does one determine that the RDF is the same? How can you show that even though the graphs are the same that the points in them are identical (ie, same exact triples)? Simplest way open up in text editors and compare them. Automate them with SPARQL. There are tools to do diffs of RDF. Not using blank nodes should help.
      • One way: turtle docs - run them through unix-like facitities - sort or something like that. Then a line by line diff. Some canonical form that can be byte by byte compared.
      • Using ntriples might make more sense since they are not multiline.
      • Are there things to filter out when doing this? Server managed triples? (example of filtering here: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/22/files#diff-d48f0429a55f2afa5045356359360d80R154) python RDF lib has isIsomorphic to test. So, filter out server managed triples and then compare the graph. Is that a good approach?
    • That we are comparing RDF should be mentioned in the test plan and the status of where that is
      • mention that we are not currently comparing RDF before/after
      • tools that we might use to do this
    • Josh can help with FCREPO-2132 Youn may add some as well. Bethany will start test section that will contain details of what actual tests we will be doing.
    • FCREPO-2134 - just add a link to the dataset and this one is done. Link to the google drive and the fcrepo-exts/sample-dataset - Nick will take this one
    • FCREPO-2164 - Andrew working on today
    • FCREPO-2178 - Nick doing today.
    • Documentation issues: 2135, 2143, 2144, 2145 - are these sufficient for communicating what this tool does and how to use it?
    • FCREPO-2143 & FCREPO-2144 sub pages (or part of a summary page). Justin will work on these on this page: https://wiki.duraspace.org/display/FEDORA4x/Import+and+Export+Tools
    • FCREPO-2145 - other then a few things (like indirect containers and remaping baseURL) it's good enough to start document. There maybe issues that change the format. Do we just want to do URL escaping for resource filenames and do that now versus later? Take care of this sprint? Yes - via url encoding/decoding. Document it as if it's fixed. Mike to create ticket about it. The entire file name will be url escaped.
    • FCREPO-2135 - will be updated to reflect where we end up on Friday. May be done on Friday.
    • FCREPO-2146 - can be rolled into the PR 22 - Esmé to work on.
    • FCREPO-2163 - Mike will work on today.
    • FCREPO-2169 & 2170 - related to one another. Not a goal for phase 1, so lower priority. Feel free to work on if you have time. Nice to have, but not necessary for phase 1.
    • FCREPO-2172 - same as above, not part of phase 1, nice to have. Keep around, but not as a requirement for phase 1.
    • FCREPO-2180 - newly added, based on earlier conversation

Everyone can take the responsibility for signing off on phase 1. Saying yes to where it works and raising a flag for where it doesn't. Please raise issues by tomorrow afternoon so we have a chance to address them.

Note that there are two dependencies on snapshots, which is not ideal. fcrepo because of Jena and then the java-client as well. Maybe after the sprint we will want to get a java-client release out so we can remove that dependency.

Wrap-up meeting on Friday at 3:30

Meeting 04 - September 9, 2016

Agenda

  • Stakeholder satisfaction; Goal: complete all phase-1 requirements and documentation
    1. Support transacting in RDF
    2. Support allowing the option to include Binaries
    3. Support references from exported resources to other exported resources
    4. Support import into a non-existing Fedora container
    5. Support export of resource and its "members" based on the ldp:contains predicate
    6. The URIs of the round-tripped resources must be the same as the original URIs
  • Retrospective
    1. What went well during the sprint cycle?
    2. What went wrong during the sprint cycle?
    3. What could we do differently to improve?
  • Penn State Sprint; September 19-23

Minutes

  • Stakeholder satisfaction
    • Requirements 1-6 were largely met
    • Requirement 6: true for relative paths
    • Possible future requirement to consider: option to delete fcr:tombstone
    • Focus on finding bugs and creating tickets, making it clear where we are now
    • Readme should include link to JIRA query
    • Bugs discussed: handling of pairtree nodes in plant patents data set; loss of properties upon import of plant patents data set; authorization errors
  • Penn State Sprint; September 19-23
    • Nick, Esme, Jon Stroop, and Andrew will participate and will touch base next week
    • Justin and Josh will check their availability and will confirm on IRC by 10am ET on Monday, September 12
    • Esme: The Penn State Sprint will focus on bags; work on bugs and other issues can happen concurrently; LDP-ICs are a priority for Sufia and Hydra
  • Other follow up
    • Mike will be working over the weekend; the expectation is that a mini-code freeze will be in place
    • Bethany will write a script to validate import and export by comparing triples, which could be provided with the utility
    • Andrew: code refactoring will happen early next week
    • Nick will draft an email notification to stakeholders and share it in a Google document
  • Retrospective
    • Andrew
      1. IRC; integration of roles
      2. Didn't finish everything
      3. Do a better job of making sure we have time available
    • Esme
      1. IRC; code review
      2. Dependency chaining; interruptions and other work
      3. More preparation at the beginning of the sprint
    • Mike
      1. Coordination working on same code base
      2. Unclear standards for fcrepo-labs code reviews
      3. Limit distractions in code reviews
    • Youn
      1. Communication
      2. Level of effort at the beginning of the sprint
      3. More preparation
    • Bethany
      1. IRC; teamwork
      2. Didn't have models to follow and know where to contribute at the beginning of the sprint
      3. Start working on scripts earlier
    • Justin
      1. IRC; group dynamics; overall what was accomplished
      2. Naive sense of commitment required; could have started sooner; learning curve with Fedora
      3. Review time commitments at kickoff; document code review process; bring data sets and examples upfront
    • Josh
      1. Daily stand ups; wiki documentation
      2. IRC somewhat distracting
      3. Clear testing phase
    • Nick
      1. Communication
      2. Better job scoping requirements; dependency chaining
      3. Blocking off calendars; assigning work; communicating schedules

 

Standups

<awoods> [Import/Export Standup]
<awoods> # Completed yesterday:
<awoods> ** Create a basic executable jar for fcrepo-import-export
<awoods> https://jira.duraspace.org/browse/FCREPO-2137
<awoods> ** Create a basic CLI framework for fcrepo-import-export
<awoods> https://jira.duraspace.org/browse/FCREPO-2138
<awoods> # Planning on completing today:
<awoods> Whatever help is useful/needed
<awoods> # Blockers / Need help with:
<awoods> None

<ruebot> [Import/Export Standup]
<ruebot> * Completed yesterday: - Stub page created for https://jira.duraspace.org/browse/FCREPO-2134 - Done https://jira.duraspace.org/browse/FCREPO-2129
<ruebot> * Planning on completing today: - https://jira.duraspace.org/browse/FCREPO-2134
<ruebot> * Blockers / Need help with: - None

<westgard> [Import/Export Standup]
<westgard> * Completed yesterday:
* dhlamb (~dhlamb@142.177.186.124) has joined
<westgard> Gathered test dataset (100 objects, ea. with a PDF file, total 55BM)
<westgard> Started batch load script to load to fcrepo (about 2/3 complete):
<westgard> https://github.com/jwestgard/plantpatents-batchload
<westgard> * Planning on completing today:
<westgard> Finish batch loader, load data to fcrepo vagrant, and export using serializer
<westgard> * Blockers / Need help with:
<westgard> None

<bseeger> [import / export standup]
<bseeger> * Completed yesterday:
* youn (8284ad7b@gateway/web/freenode/ip.130.132.173.123) has joined
<bseeger> I've been lurking mostly, glad to help where I can, but not sure what that is right now, though I'm sure there will be more later.
<bseeger> * Planning on completing today:
<bseeger> helping with the test doc
<bseeger> * Blocker / Need help with:
<bseeger> Nothing right now. Been wondering though, what about fedora triples and maintaining them on an import - ie, restoring based on export. Probably not part of first phase, but should be designed in.

<youn> * Completed yesterday: initial draft of test plan for import export sprint (edits and feedback welcome!)
* westgard has quit (Ping timeout: 250 seconds)
* ajs6f (~ajs6f@d-137-54-155-242.dhcp.virginia.edu) has joined
<youn> * Planning on completing today: updating test plan; starting work on components
<youn> * Blockers / Need help with: feedback on test plan

<escowles> [import/export standup]:
<escowles> * completed yesterday: added stub readme and maven execution plugin, started on export utility
<escowles> * today: just merged #4 (command-line parsing), plan on finishing basic export utility and creating tickets for obvious improvements
<escowles> * blockers: none

<justinsimpson> [Import/Export Standup]
<justinsimpson> * Completed yesterday: - attended kick off meeting (first hour)
<justinsimpson> * Planning on completing today: - setting up test environment and populating with test data
<justinsimpson> * Blockers / Need help with: - None

<ruebot> [Import/Export Standup]
<ruebot> * Completed yesterday:
<ruebot> - A significant amount of meetings
<ruebot> * Planning on completing today:
<ruebot> - https://github.com/fcrepo4-labs/fcrepo-import-export/pull/6 -- testing
<ruebot> - contribute to https://jira.duraspace.org/browse/FCREPO-2134
<ruebot> * Blockers / Need help with:
<ruebot> - nada

<awoods> [Import/Export Standup]
<awoods> * Completed yesterday:
<awoods> - Review of: (Create a test plan) https://jira.duraspace.org/browse/FCREPO-2132
<awoods> * Planning on completing today:
<awoods> - Review and testing of anything that is "In Review"
<awoods> * Blockers / Need help with:
<awoods> - Please let me know if you need help, should we schedule a call for Thu? Fri?

<barmintor_> * yesterday: split the i/e project up into a multimodule so that we could work on more than one component at once
<barmintor_> * today: working on FCREPO-2130
<barmintor_> * blockers: none that I know of

<youn> [Import/Export Standup]
<youn> * Completed yesterday: test resources - binaries
<youn> * Planning on completing today: containers with RDF and binaries
<youn> * Blockers / Need help with: none

<justinsimpson> [Import/Export Standup] (Justin Simpson)
<justinsimpson> * Completed yesterday:
<justinsimpson> set up a local test environment
<justinsimpson> * Planning on completing today:
<justinsimpson> try an initial test of https://github.com/fcrepo4-labs/fcrepo-import-export/pull/6
<justinsimpson> * Blockers / Need help with:
<justinsimpson> none

<westgard> [Import/Export Standup]
<westgard> * Completed yesterday:
<westgard> Finished loading of plant patents dataset that can be used for testing.
<westgard> * Planning on completing today:
<westgard> A few refinements, then export XML and make available by some means agreeable to all (size is around 60 MB).
<westgard> * Blockers / Need help with:
<westgard> Just need to know how we want to share this dataset.

<bseeger> [Import/Export Standup]
<bseeger> * completed yesterday: looked at test plan, added comments
<bseeger> * planning on completing today: setup test env
<bseeger> * blockers / need help with: just not quite sure what to do yet, but helping here and there.

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa> * Completed yesterday:
<mikeAtUVa> - Wiki documentation stub: https://jira.duraspace.org/browse/FCREPO-2136
<mikeAtUVa> - Explored camel serializer
<mikeAtUVa> * Planning on completing today:
<mikeAtUVa> - https://jira.duraspace.org/browse/FCREPO-2127 or something more useful
<mikeAtUVa> * Blockers / Need help with:
<mikeAtUVa> - not sure whether FCREPO-2127 is still relevant or where to put the documentation
<mikeAtUVa> - distracted by https://jira.duraspace.org/browse/FCREPO-2086
<mikeAtUVa> - not sure what to work on next

< ruebot> [Import/Export Standup]
< ruebot> * Completed yesterday:
< ruebot>   - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/6
< ruebot>   - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/7
< ruebot>   - Created issue for recursive binary export https://jira.duraspace.org/browse/FCREPO-2152
< ruebot> * Planning on completing today:
< ruebot>   - Test and merge https://github.com/fcrepo4-labs/fcrepo-import-export/pull/8
< ruebot>   - Evaluate where we're at with phase 1
< ruebot>   - Do a better job at assigning work
< ruebot> * Blockers / Need help with:
< ruebot>   - None

< awoods> [Import/Export Standup]
< awoods> * Completed yesterday:
< awoods> - Code reviews/testing
< awoods> * Planning on completing today:
< awoods> - Code reviews/testing
< awoods> - Importing test data-sets (hint)
< awoods> * Blockers / Need help with:
< awoods> - None

<youn> [Import/Export Standup]
<youn> * Completed yesterday: reviewed Fedora RESTful HTTP API documentation and W3C Linked Data Platform recommendation
<youn> * Planning on completing today: script to create (1) containers with RDF and (2) containers with RDF and binaries, to include with test plan
<youn> * Blockers / Need help with: if not included elsewhere and consensus on need, checksum comparison tool to compare checksums (a) before and (b) after export and (c) before and (d) after import, to include with test plan

<escowles> [import/export standup]
<escowles> * yesterday: got basic export and saving binaries only if configured PRs merged, started working on recursive export
<escowles> * today: 4hrs of meetings, so not too much — will try to finish recursive retrieval and make sure there are tickets for some of the good ideas that have come up (e.g. using IOUtils.copyLarge when appropriate)
<escowles> * blockers: none

<bseeger> [Import/Export Standup]
<bseeger> * Completed yesterday: some testing of PR to setup test env.
<bseeger> * Planning on completing today: in many, many meetings, so probably not much for I/E
<bseeger> * Blockers / Need help with: nothing, currently

<justinsimpson> [Import/Export Standup]
<justinsimpson> * Completed yesterday:
<justinsimpson> very little, got diverted
<justinsimpson> * Planning on completing today:
<justinsimpson> build and test https://github.com/fcrepo4-labs/fcrepo-import-export maybe look at pr8?
<justinsimpson> * Blockers / Need help with:
<justinsimpson> wondering how to find the test data westgard made?

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa> * Completed yesterday: most of https://jira.duraspace.org/browse/FCREPO-2127
<mikeAtUVa> * Planning on completing today: the rest of that, and catching up with the actual softare development
<mikeAtUVa> * Blockers / Need help with: noting related to this sprint

<westgard> [Import/Export Standup]
<westgard> * Completed yesterday:
<westgard> - Some refinements to test data
<westgard> * Planning on completing today:
<westgard> - Finalize the metadata in the test dataset (such that it will be real data that one can meaningfully query)
<westgard> - Give access to test data
<westgard> * Blockers / Need help with:
<westgard> - still need clarity on how and where to provide access to the test data (sorry if this should be obvious); I will be in a lot of meetings today starting now, but will try to get to this earlier rather than later

<barmintor> [Import/Export Standup] Yesterday: refactored project into yet more mvn modules
<barmintor> [Import/Export Standup] Today: PR #8, then back to 2130
<barmintor> [Import/Export Standup] Blockers: none that I am aware of

 



08:15:52 < ruebot> [Import/Export Standup]
08:15:56 < ruebot> * Completed yesterday:
08:15:59 < ruebot>   - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/8
08:16:04 < ruebot>   - Tested and reviewed https://github.com/fcrepo4-labs/fcrepo-import-export/pull/9
08:16:08 < ruebot>   - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/10
08:16:13 < ruebot>   - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/11
08:16:17 < ruebot>   - Tested and reviewed https://github.com/fcrepo4-labs/fcrepo-import-export/pull/14
08:16:24 < ruebot> * Planning on completing today:
08:16:27 < ruebot>   - Test and merge https://github.com/fcrepo4-labs/fcrepo-import-export/pull/12
08:16:32 < ruebot>   - Review and merge https://github.com/fcrepo4-labs/fcrepo-sample-dataset/pull/6
08:16:35 < ruebot>   - Review Fedora 4 Import Export Utility Tests
08:16:38 < ruebot>   - Determine whether https://jira.duraspace.org/browse/FCREPO-2127 is good to close
08:16:43 < ruebot> * Blockers / Need help with:
08:16:47 < ruebot>   - None

09:00:58 < awoods> [Import/Export Standup]
09:00:59 < awoods> * Completed yesterday:
09:00:59 < awoods> -- Code reviews
09:00:59 < awoods> -- Add Standard Plugins: https://jira.duraspace.org/browse/FCREPO-2161
09:00:59 < awoods> -- Fix broken executable jar: https://jira.duraspace.org/browse/FCREPO-2160
09:00:59 < awoods> * Planning on completing today:
09:01:01 < awoods> -- Create integration test framework: https://jira.duraspace.org/browse/FCREPO-2155
09:01:03 < awoods> -- Use top-level dependency management: https://jira.duraspace.org/browse/FCREPO-2162
09:01:05 < awoods> * Blockers / Need help with:
09:01:07 < awoods> -- None

09:05:26 < escowles> [import/export standup]
09:06:04 < escowles> * yesterday: finished binary descriptions
09:06:48 < escowles> * today: export logging and error handling, maybe look at basic import functionality if barmintor's not working that today
09:07:11 < escowles> * blockers: none

09:21:47 < youn> [Import/Export Standup]
09:22:03 < youn> * Completed yesterday: generated OWL data; uploaded to shared Google folder
09:22:21 < youn> * Planning on completing today: script to load OWL files and binaries (thanks for pointing out the scripts on fcrepo-labs)
09:22:28 < youn> * Blockers / Need help with: none

09:48:54 < justinsimpson> [Import/Export Standup]
09:49:00 < justinsimpson> [Import/Export Standup]
09:49:00 < justinsimpson> * Completed yesterday:
09:49:10 < justinsimpson> experimented with adding sample data to my fedora test instance
09:49:17 < justinsimpson> tried running the export utility, but I didn't see any output
09:49:24 < justinsimpson> * Planning on completing today:
09:49:31 < justinsimpson> successfully run the export utility
09:49:41 < justinsimpson> looking at the sample data others have provided so far
09:49:46 < justinsimpson> contintue to make my own sample data
09:49:52 < justinsimpson> * Blockers / Need help with:
09:49:59 < justinsimpson> not sure yet how to use the samples youn put in the shared google folder

09:54:39 < bseeger> [Import/Export Standup]
09:55:15 < bseeger> * Completed yesterday:  PR code testing, fixing 2149
09:55:33 < bseeger> * Planning on completing today: more PR testing, not sure what else
09:55:45 < bseeger> * Blockers / Need help with: none

10:04:35 < westgard> [Import/Export Standup]
10:04:37 < westgard> * Completed yesterday:
10:04:39 < westgard>   - pull request for some sample data (200 objects)
10:04:41 < westgard> * Planning on completing today:
10:04:43 < westgard>   - sort out some issues with sparql update errors in my sample data loader
10:04:45 < westgard>   - add external references to the sample dataset
10:04:47 < westgard> * Blockers:
10:04:49 < westgard>   - Weltschmertz and the imponderableness of local master branches :-)

 <ruebot> [Import/Export Standup]

<ruebot> * Completed yesterday:
<ruebot> - https://github.com/fcrepo4-labs/fcrepo-import-export/pull/18
<ruebot> .* Planning on completing today:
<ruebot> - Review, test, merge, whatever comes in
<ruebot> - Complete https://jira.duraspace.org/browse/FCREPO-2132
<ruebot> - Spend the entire afternoon in meetings
<ruebot> * Blockers / Need help with:
<ruebot> - None

<awoods> [Import/Export Standup]
<awoods> * Completed yesterday:
<awoods> - Create integration test framework: https://jira.duraspace.org/browse/FCREPO-2155
<awoods> * Planning on completing today:
<awoods> - Use top-level dependency management: https://jira.duraspace.org/browse/FCREPO-2162
<awoods> * Blockers / Need help with:
<awoods> - None

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa> * Completed yesterday:
<mikeAtUVa> - Took a first pass at https://jira.duraspace.org/browse/FCREPO-2165
<mikeAtUVa> - Reviewed https://jira.duraspace.org/browse/FCREPO-2155
<mikeAtUVa> * Planning on completing today:
<mikeAtUVa> - Addressing comments for https://jira.duraspace.org/browse/FCREPO-2165
<mikeAtUVa> - https://jira.duraspace.org/browse/FCREPO-2163, unless @ruebot says no and tasks me with something else
<mikeAtUVa> * Blockers / Need help with:
<mikeAtUVa> - None

<escowles> [import/export standup]
<escowles> * yesterday: updated exporter to export binary descriptions and log files exported, rough draft of import client
<escowles> * today: get basic import client working and create tickets for follow-on work
<escowles> * blockers: none

<bseeger> [Import/Export Standup]
<bseeger> * Completed Friday:
<bseeger> - finished fcrepo-2149, participated in sprint phone call
<bseeger> * Planning on completing today:
<bseeger> - probably help test PR's
<bseeger> * Blockers / Need help with:
<bseeger> - None

<justinsimpson> [Import/Export Standup]
<justinsimpson> * Completed yesterday:
<justinsimpson> (really Friday) started work on https://jira.duraspace.org/browse/FCREPO-2127
<justinsimpson> * Planning on completing today:
<justinsimpson> completing 2127
<justinsimpson> * Blockers / Need help with:
<justinsimpson> none

<westgard> [Import/Export Standup]
<westgard> * Completed:
<westgard> nearly there on FCREPO-2166 and 2167, have rewritten the loader for sample data for this purpose, improving the design quite a lot: https://github.com/jwestgard/plantpatents-batchload/blob/develop/pp-load.py
<westgard> * Planning on completing today:
<westgard> should be able to wrap up these two tickets and submit a pull request for new sample data
<westgard> * Blockers:
<westgard> None

<youn> [Import/Export Standup]
<youn> * Completed yesterday: finalized test plan; wrote script to modify URIs in lubm owl files for ingest (following Esme's suggestion)
<youn> * Planning on completing today: loading lubm rdf; adding binaries for pubs and maybe ldp properties for certain ub ontology properties; redoing multi-file binaries to get to manageable size
<youn> * Blockers / Need help with: tbd

<escowles>  [import/export standup]
<escowles> * yesterday: got basic import working (updated fcrepo-java-client to support some needed features)
<escowles>* today: tests for basic import & a few meetings
<escowles> blockers: none, other than the meetings

<youn> [import/export standup]
<youn> * yesterday: figured out how to use RESTful HTTP API
<youn> * today: finish scripts and load lubm data
<youn> * blockers: none (known yet)

<bseeger> [Import / Export Standup]
<bseeger> * Completed Yesterday: no sprint things, though I am watching PR's
<bseeger> * Planning on completing today:
<bseeger> - load test data that's available
<bseeger> * Blockers / Need help with:
<bseeger> - none

<ruebot>    [Import / Export Standup]
<ruebot>    * Completed Yesterday:
<ruebot>    - https://github.com/fcrepo4-labs/fcrepo-import-export/pull/18
<ruebot>    - Tested, reviewed, merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/21
<ruebot>    - https://jira.duraspace.org/browse/FCREPO-2132 -- Added wiki page; are we good to close that ticket, or do we leave it open for future sprints/phases?
<ruebot>    * Planning on completing today:
<ruebot>    - https://jira.duraspace.org/browse/FCREPO-2178
<ruebot>    * Blockers / Need help with:

<awoods>    [Import/Export Standup]
<awoods>    * Completed yesterday:
<awoods>    - Remove non-test dependencies on fcrepo: https://jira.duraspace.org/browse/FCREPO-2177
<awoods>    - Add error handling and reporting failures: https://jira.duraspace.org/browse/FCREPO-2148
<awoods>    * Planning on completing today:
<awoods>    - Add support for config file: https://jira.duraspace.org/browse/FCREPO-2164
<awoods>    * Blockers / Need help with:
<awoods>    - None

<justinsimpson> [import/export standup]
<justinsimpson>    * Completed yesterday:
<justinsimpson>    completed
<justinsimpson>    * Today:
<justinsimpson>    would like to test import utility. Many meetings, which will limit my time today
<justinsimpson>    Blockers:
<justinsimpson>    none

<westgard>    [Import/Export Sprint Standup]
<westgard>    * Completed yesterday:
<westgard>    troubleshot and wrote up documentation on the system-generated namespace issue;
<westgard>    * Planning on completing today:
<westgard>    FCREPO-2166 and 2167
<westgard>    * Blockers:
<westgard>    None

<youn>    [Import/Export Standup]
<youn>    * Completed yesterday: scripts to process and load lubm data
<youn>    * Planning on completing today: sparql queries; help with test plan if needed
<youn>    * Blockers / Need help with: load script halts (performance issue?)

<bseeger>    [Import / Export Standup]
<bseeger>    * Completed Yesterday: meeting
<bseeger>    * Planning on completing today:
<bseeger>    - add specifics about tests to tests plans; load test data if possible.
<bseeger>    * Blockers / Need help with:
<bseeger>    - none

<ruebot> [Import / Export Standup]
<ruebot> * Completed Yesterday:
<ruebot>    - Tested, reviewed, merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/21
<ruebot>    * Planning on completing today:
<ruebot>    - https://jira.duraspace.org/browse/FCREPO-2178
<ruebot>    - Review test https://github.com/fcrepo4-labs/fcrepo-import-export/pull/25
<ruebot>    - Take Youn's test data for a spin
<ruebot>    * Blockers / Need help with:
<ruebot>    - My schedule :-(

<awoods>    [Import/Export Standup]
<awoods>    * Completed yesterday:
<awoods>    - Remove URISyntaxExceptions: https://jira.duraspace.org/browse/FCREPO-2183
<awoods>    - Add support for config file: https://jira.duraspace.org/browse/FCREPO-2164
<awoods>    * Planning on completing today:
<awoods>    - Response to code review comments on FCREPO-2164    09:43
<awoods>    - Testing datasets
<awoods>    * Blockers / Need help with:
<awoods>    - None

<justinsimpson>    [Import/Export Standup]
<justinsimpson>    Yesterday:
<justinsimpson>    no progress, too many meetings
<justinsimpson>    Today:
<justinsimpson>    starting with
<justinsimpson>    planning to finish #2143, #2144, #2145
<justinsimpson>    Blockers/Questions:
<justinsimpson>    none

<westgard>    [Import/Export Standup]
<westgard>    * completed yesterday/this morning:
<westgard>    - pull request that should resolve FCREPO-2166, 67, & 68
<westgard>    * planning on completing today:
<westgard>    - try out youn's data and import shell script
<westgard>    - verify and if necessary file a bug report for an issue discovered yesterday by which a typo in SPARQL update can cause binaries to disappear (404 not found though they are in fact still there)

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa>    * Completed yesterday:
<mikeAtUVa>    - https://github.com/fcrepo4-labs/fcrepo-import-export/pull/17
<mikeAtUVa>    * Planning on completing today:
<mikeAtUVa>    - https://jira.duraspace.org/browse/FCREPO-2163
<mikeAtUVa>    - https://jira.duraspace.org/browse/FCREPO-2181
<mikeAtUVa>    * Blockers / Need help with:
<mikeAtUVa>    - None

<westgard>    [Import/Export Sprint]
<westgard>    * Completed yesterday:
<westgard>    - FCREPO-21{66,67,68} pull request
<westgard>    - Wrote up and filed FCREPO-2186 (error caused by malformed SPARQL)
<westgard>    - Tested LUBM data load scripts
<westgard>    * Planning on completing today:
<westgard>    - FCREPO-2189 (add checksum lists, README, etc. to dataset)
<westgard>    - build and run import/emport tools
<westgard>    * Blockers
<westgard>    - None, but I may be a few minutes late to the 3:30 ET meeting

<youn>    [Import/Export Standup]
<youn>    * Completed yesterday: added ub types and consolidated sparql queries for lubm data set
<youn>    * Planning on completing today: add indirect containers for publications; try using sleep to be able to load and update data
<youn>    * Blockers / Need help with: none (known yet)

<mikeAtUVa> [Import/Export Standup]
<mikeAtUVa>    * Completed yesterday:
<mikeAtUVa>    - https://jira.duraspace.org/browse/FCREPO-2163
<mikeAtUVa>    - https://jira.duraspace.org/browse/FCREPO-2181
<mikeAtUVa>    * Planning on completing today:
<mikeAtUVa>    - testing, bugfixes, documentation improvements
<mikeAtUVa>    * Blockers / Need help with:
<mikeAtUVa>    - None

<escowles> [import/export standup]:
<escowles>    * yesterday: worked on uri translation (fcrepo-2180)
<escowles>    * today: get that done, fix external content, then look at other bugs
<escowles>    * blockers: none

<awoods>    [Import/Export Standup]
<awoods>    * Completed yesterday:
<awoods>    - Reported two bugs:
<awoods>    ** ERROR on export of dataset: https://jira.duraspace.org/browse/FCREPO-2187
<awoods>    ** ERROR on import of dataset: https://jira.duraspace.org/browse/FCREPO-2188
<awoods>    * Planning on completing today:
<awoods>    - Testing
<awoods>    * Blockers / Need help with:
<awoods>    - None

<bseeger>    [Import/Export Standup]
<bseeger>    * Completed Yesterday:
<bseeger>    - Start of test part of test plan
<bseeger>    * Planning on completing today:
<bseeger>    - Look at test plan and merge test section with what Josh Westgard added.
<bseeger>    - May write a script to attempt to verify that export/import worked. This would
<bseeger>    be something that would look at exported n-triple files and sort & diff them w/ fedora files via very basic
<bseeger>    mechanisms.
<bseeger>    * Blockers / Need help with:
<bseeger>    - None

<justinsimpson> [Import/Export Standup]
<justinsimpson>    * Completed yesterday:
<justinsimpson>    worked on https://jira.duraspace.org/browse/FCREPO-2143 and
<justinsimpson>    https://jira.duraspace.org/browse/FCREPO-2144
<justinsimpson>    * Planning on completing today:
<justinsimpson>    will complete updates on wiki for
<justinsimpson>    https://jira.duraspace.org/browse/FCREPO-2143 https://jira.duraspace.org/browse/FCREPO-2144 https://jira.duraspace.org/browse/FCREPO-2145
<justinsimpson>    before the meeting this afternoon
<justinsimpson>    * Blockers / Need help with:
<justinsimpson>    none