Date

29 Aug 2019 from 14:00-15:00 UTC

Location: https://lyrasis.zoom.us/my/dspace (Meeting ID: 502 527 3040).

• More connection options available at DSpace Meeting Room

Agenda

• (15 mins) Developer Stand Up - Developers give brief updates on their effort (or their team's effort).

  • Update/see "Current Work" section below based on your status. Please feel free to update prior to meeting.
  • Please highlight any new work (needing reviews/testing), any blockers (for you), and any discussion topics you may have.
• (30 mins) General Discussion Topics
  
  1. DSpace 7 Estimation Process Overview. Starting to locate volunteers & plan out when these estimates will take place, etc.
  2. (Topic #2)
• (15 mins) Planning for next week
  
  • Assigning PRs for Review
  • Next tasks from Development Planning Spreadsheet

Attendees

• Art Lowel (Atmire) 

• Andrea Bollini (4Science)  

• Tim Donohue 

• Heather Greer Klein (unavailable)

• Lieven Droogmans 

• Mark H. Wood
• Giuseppe Digilio (4Science) 
• Ben Bosman
• Pascal-Nicolas Becker (unavailable)
• Chris Wilper
• Terrence W Brady
• Paulo Graça 
• Dimitris Pierrakos (unavailable)
• Julius Gruber (unavailable)
• Laura Henze

Current Work

Tickets / PRs In Progress

1. (Angular) Adding Accessibility via Travis CI  https://github.com/DSpace/dspace-angular/pull/356 (work in progress) (Lower priority)
2.  (Angular Bug) https://github.com/DSpace/dspace-angular/issues/368 ( Art Lowel (Atmire) )
3. (REST Contract) Edit Homepage news: https://github.com/DSpace/Rest7Contract/pull/45 (Ben Bosman  - has outstanding questions/comments) (Lower priority)
4. (REST) DS-4043: Revisit the security layer of the submission  (work in progress) Andrea Bollini (4Science)
5.  (REST Contract) Group and eperson management: https://github.com/DSpace/Rest7Contract/pull/41 (Waiting on updates fromBen Bosman )

PRs Needing Review

1. (REST Contract) DS-4317 bundles in REST https://github.com/DSpace/Rest7Contract/pull/72 ( Tim Donohue - minor tweaks suggested,  Andrea Bollini (4Science) )
2. (REST Contract) upload bitstream to archived item - update https://github.com/DSpace/Rest7Contract/pull/71 ( Tim Donohue, Andrea Bollini (4Science) )
3. (REST) Pagination bug with withdrawn items: https://github.com/DSpace/DSpace/pull/2406 (Dimitris Pierrakos , Ben Bosman - Feedback provided)
4. (REST) Issue when community has multiple dc.title values https://github.com/DSpace/DSpace/pull/2486 ( Tim Donohue , Andrea Bollini (4Science) )
5. (REST) Oai harvesting setup https://github.com/DSpace/DSpace/pull/2491 (Tim Donohue, Andrea Bollini (4Science))
6. (Angular) (Entities) Deleting relationships: https://github.com/DSpace/dspace-angular/pull/402 (Paulo Graça - will test again, Tim Donohue )
7. (Angular) Move Item Component: https://github.com/DSpace/dspace-angular/pull/335 (Giuseppe Digilio (4Science) - reviewed again and provided feedback, Tim Donohue - REREVIEW)
8. (Angular) Item-Collection Mapper:  https://github.com/DSpace/dspace-angular/pull/348 ( Tim Donohue - REREVIEW, Giuseppe Digilio (4Science) )
9. (Angular) Shibboleth integration support: https://github.com/DSpace/dspace-angular/pull/429  (Julius running into an error with 'yarn start' only) (Giuseppe Digilio (4Science) reviewed again and provided feedback, Fernando FCT/FCCN, Paulo Graça )
10. (Angular) Submission Miscellaneous fixes: https://github.com/DSpace/dspace-angular/pull/432 (Art Lowel (Atmire), Julius Gruber , Tim Donohue )
11. (Angular) UI Language Cookie https://github.com/DSpace/dspace-angular/pull/443 ( Tim Donohue, Paulo Graça , NEEDS SECOND REVIEWER)
12. (Angular) Convert i18n files to JSON5 format https://github.com/DSpace/dspace-angular/pull/439 (Tim Donohue, NEEDS SECOND REVIEWER)
13. (Angular) Search Performance optimizations #2 https://github.com/DSpace/dspace-angular/pull/437 (Tim Donohue, Giuseppe Digilio (4Science) approved)
14. (Angular) bitstream format registries https://github.com/DSpace/dspace-angular/pull/455 (Tim Donohue ,  Art Lowel (Atmire))
15. (Angular) Fix the labels on edit collection and community pages https://github.com/DSpace/dspace-angular/pull/459 (Giuseppe Digilio (4Science) approved, Tim Donohue )
16. Indirect entity refs during CSV import https://github.com/DSpace/DSpace/pull/2471 (Ben Bosman -  reviewed and provided feedback, Tim Donohue )
17. (Backend) Solr 7 fixes for upgrading to DSpace 7 https://github.com/DSpace/DSpace/pull/2393 (Chris Wilper , NEEDS SECOND REVIEWER) 
18. (NEW) (Angular) Redirecting user to same page after login https://github.com/DSpace/dspace-angular/pull/467 (Art Lowel (Atmire), Giuseppe Digilio (4Science))
19. (NEW) (REST) Spring security for createAndReturn with parent id https://github.com/DSpace/DSpace/pull/2489 (Tim Donohue, NEEDS SECOND REVIEWER) 
20. (NEW) (Backend) Upgrade to Solr 7:  support sharded statistics https://github.com/DSpace/DSpace/pull/2495 (Chris Wilper, NEEDS SECOND REVIEWER)

PRs Merged this week!

1.  (REST Contract) Collecting statistics - https://github.com/DSpace/Rest7Contract/pull/63
2.  (REST Contract) Update bitstream properties and format https://github.com/DSpace/Rest7Contract/pull/68
3.  (REST) implement upload bitstream to archived item https://github.com/DSpace/DSpace/pull/2473

Blocked

1. (Blocked PRs go here)

Delayed / Needs Discussion

1. (REST) Scripts & Processes endpoint: https://github.com/DSpace/Rest7Contract/pull/17
   1. How do we move this forward? In our Jan 24 meeting, Tim noted a risk of "scope creep" with this feature suggestion. This idea has been tabled since then.
   2. In July 25 meeting, Atmire said they'd come back with notes on the proposed backend implementation.
2. Managing Authorization info in Angular UI: How to pass Authorization rights (i.e. logged in user's access rights) from REST API to Angular?  See for example: https://github.com/DSpace/dspace-angular/issues/393
   1. Can this be achieved via passed HAL "_links" (e.g. the existence of an "edit" link in REST response means you must have Edit rights)?
   2. In July 25 meeting, we noted this probably cannot be resolved with just one simple solution. May need to look at different options for different scenarios
      1. Also likely to need to store/cache a user's Groups in UI layer, as some areas (e.g. Administrative) require knowledge of user group membership
3. Initial Performance Testing from Chris.
   1. https://cwilper.github.io/dspace-perftest/
4. (REST Contract) Edit Homepage News: https://github.com/DSpace/Rest7Contract/pull/45
   
   1. Delayed until after Preview release. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior.  Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata.  Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.  
   2. Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question.  May allow for more CMS-lite behavior in the future
   3. Timeline for this is uncertain.  Possibly in 7 or 8. May depend on how/whether it can be scoped.
5. Concurrency in DSpace 7 (or 8).  What do we want to do when multiple editors are editing the same object?  Needs further analysis regarding implementation details
   1. We've decided (in meeting on March 7, 2019) to use ETags to implement concurrency. REST Contract notes on ETags: https://github.com/DSpace/Rest7Contract#etags--conditional-headers
   2. ETags only update of the two fields match. If someone edits first, your edit would fail and you would get a fail response (422?)
   3. ETags seems to have broader support in other REST APIs.  Recommended also by both Art and Andrea.

6. Improve/Re-enable End To End (e2e) Testing. Could there be opportunities to use Travis CI + Docker Compose for testing of Angular?? https://github.com/DSpace/dspace-angular/issues/453#issuecomment-519672141

   • Docs: https://docs.travis-ci.com/user/docker/#using-docker-compose
   • Blogpost on how to do it: http://elliot.land/post/using-docker-compose-on-travis-ci
   • An example travis.yml file: https://github.com/Ortus-Solutions/docker-buildfiles/blob/master/.travis.example.yml

Notes

• Detailed walkthrough / discussion of DSpace 7 Estimation Process
  • Talked through details on wiki page around the overall process itself
  • KEEP IN MIND: This first time around we are only estimating tasks that are well defined.  Anything in the Development Planning Spreadsheet marked currently as "NEEDS MORE INFO" is not included in this initial estimation process.  We will work to better define those tasks in coming weeks, and add them to a second round of estimation.
  • Walked through the first REST API spreadsheet, talking about how the process will work for individual developers
  • Feedback on general process
    • Since we don't have a specific developer pool or specific developers, this makes some estimates more challenging.  Usually you estimate based on known developer skill sets, etc
    • We don't even know if we have senior vs junior developers
    • Tim notes, maybe that's something we simply need to write into the estimates themselves (as assumptions).  More complex tasks may need to be assumed to be implemented by a senior developer (and we should clearly note that as part of the estimate).  Less complex tests should likely be estimated with a junior developer in mind, but if a senior developer picks it up, it could go much quicker than expected.
    • TODO: This is also something we can document better in our Estimation process Wiki page.  Tim will do so.
  • Feedback on spreadsheet
    • Andrea asks what about Integration / Unit Tests?  How do those get estimated
    • Tim says assumption was they'd be part of the development process, since we are doing TDD. But we could separate them into a third column, if it makes it more clear
    • Andrea says that we should either create a new column for creating tests, or ensure all estimators also have them in mind.
    • TODO: Tim will update our Estimation Process wiki page to make it clear that Tests need estimating too.  Will also think more about whether a new column seems like a good direction here.
  • Other Feedback
    • Lieven notes that we need to determine a way to eventually turn this into timelines
    • Tim agrees. Timelines depend on both estimates & resources though.  So, as we are doing estimates, we will also need to get a better handle on resources.  More resources can obviously decrease timelines, even if estimates show a lot of work left to do.
    • Lieven also notes that as we look at timelines, we need to be careful to consider dependencies among tasks.  Most tasks start at the REST Contract level, then REST implementation, then Angular design & implementation.   Need to ensure we schedule tasks to keep this in mind, as timelines can also be affected by dependencies that hold other development back / keep other developers from doing their work.
  • Lieven notes that Atmire team likes approach, feels they could start immediately (and provide feedback as they go).  Tim will create developer spreadsheets for anyone ready to start
  • Andrea notes that 4Science would also love to contribute. First few weeks of Sept might be different (per their own schedule), but week of Sept 16 currently looks open.
  • Andrea (and Tim) highlight we would also like to see if we can find estimators who are not  from Atmire or 4Science. This process works well even with developers who are unsure about their estimates.
    • ALL: Please consider whether you can chip in on some of these estimates as an estimator!  Review the spreadsheets, and let Tim know next week (Sept 5).
• Next Week
  • Touch back on the Estimation Process.  Anyone have feedback or questions now that you've looked more at the spreadsheets?
  • Determine our list of estimators. Finalize who is doing estimates
  • Finalize the estimation timeline (will it take estimators 1 week? two weeks? three weeks?)
  • Start to schedule first Estimation Meeting(s) based on those timelines