Date

07 May 2020 from 14:00-15:00 UTC

Location: https://lyrasis.zoom.us/my/dspace (Meeting ID: 502 527 3040).

• More connection options available at DSpace Meeting Room

Agenda

• (15 mins) Developer Stand Up - Developers give brief updates on their effort (or their team's effort).

  • Update/see "Current Work" section below based on your status. Please feel free to update prior to meeting.
  • Please highlight any new work (needing reviews/testing), any blockers (for you), and any discussion topics you may have.
• (30 mins) General Discussion Topics
  
  1. (10 mins) Tim summarizes next steps for BTE vs Live Import 
     
     1. BTE framework was deprecated in 6.0, with Live Import framework selected as the replacement. 
     2. As discussed previously, the "Metadata Suggestions" PR (https://github.com/DSpace/DSpace/pull/2712) includes useful enhancements to Live Import functionality (from how it existed in 6.x).  However, this doesn't quite replace existing BTE functionality.
     3. Tim Donohue's recommendation is to start from a PR that implements a more simplistic implementation of Live Import that can be used to swap out BTE more directly.  Would this be possible?
        1. The idea would be to start small here with a basic PR that disables/replaces BTE from REST API in favor of Live Import. The focus would be on replacing/refactoring the two `dspace-server-webapp` classes that touch BTE as documented in this JIRA comment
        2. We would then follow that up with additional PRs to add/enhance functionality of Live Import to better align with (other) existing features in BTE.  Including PR#2712, but also including support for additional formats/services and perhaps improved command-line support (if deemed necessary)
     4. Ben Bosman has suggested that we might consider creating a BTE "plugin" for Live Import to help with the transition.  This will allow us to build all DSpace 7 features using the Live Import framework, while (temporarily) allowing BTE features/functions to still be able to work via Live Import. (Tim notes that this should only be temporary to avoid short term feature loss.  The end goal should still be one framework.)
     5. If others feel necessary, a separate meeting specific to BTE & Live Import could be scheduled to talk through the details of this change.
  2. (10 mins) (REST) (beta 3) Subresources should obey access restrictions https://github.com/DSpace/DSpace/pull/2726 
     1. Tim Donohue added feedback to the PR based on testing/analysis of Spring Security annotations: https://github.com/DSpace/DSpace/pull/2726#issuecomment-622581818
  3. (10 mins) How to move forward on Controlled Vocabulary feature: https://github.com/DSpace/DSpace/pull/2743
     1. Ben Bosman added feedback on feature gaps between DSpace 6 and DSpace 7 with regards to Controlled Vocabulary. How can we work to address these gaps in upcoming PR(s)?
     2. Early notes on differences in behavior: Controlled Vocabularies and Authority Control in DSpace 7
     3. This topic may be short on time, however if someone(s) is willing to draft a proposal for addressing these gaps, we can revisit next week.
  4. Tabled Topics (will not be discussed unless time allows)
     1. Restricted endpoints are sometimes the only HAL link path to public endpoints ()
        1. Some endpoints don't allow you to GET them, but do have publicly accessible child endpoints (e.g. registrations, resourcepolicies, …). To get to those child endpoints the UI can't discover them because the parent endpoint returns a 4xx status code. We should consider a way to prevent a regular response but allow retrieving the HAL links
     2. Finalize / approve the initial list of all authorization features which we should implement for the /api/authz/features REST endpoint.  This list of features should be limited to only features which are required to enable/disable User Interface functionality. (In other words, we can always add more features in the future.  We just need to approve the list necessary for 7.0)
        1. Review current spreadsheet (from Andrea Bollini (4Science) ) : https://docs.google.com/spreadsheets/d/1182LcD_WqIZRbUGWpLtBw0aOMR9jhbOVB7GZqtTpR9A/edit?usp=sharing 
           1. Art Lowel (Atmire) : I don't see any immediate issues with the current set of features, but I would prefer a consistent naming scheme. I'd use canDoSomething for everything
     3. PROPOSAL FROM 4SCIENCE https://wiki.lyrasis.org/x/jwUoCw
        1. Revisiting "Collection dropdown in submission" issues (how should we address these?  Beta 3 has several other submission tickets, should we include work on these?)
        2. Known performance issues: https://github.com/DSpace/dspace-angular/issues/487 (NOTE: These are still reproducible on the demo site.  If you load a new submission, there's a 4-5 second pause before the page becomes usable)
        3. Changing Collection doesn't work when new collection has a different form definition: https://github.com/DSpace/dspace-angular/issues/621
           1. If I recall correctly, Collection item templates also are not working right in this scenario.
           2. And I don't recall whether previously entered metadata gets saved or cleared out when collections have different metadata requirements.
           3. Since it's possible to start a new submission via a default Collection, this increases the likelihood of encountering these issues (if the deposit starts in the wrong collection).
        4. somehow related to this discussion Creation of DSpace Entities
• (15 mins) Planning for next week
  
  • Claim tasks for next week from DSpace 7 Release Plan spreadsheet
  • Assigning PRs for Review

Attendees

• Art Lowel (Atmire) 

• Andrea Bollini (4Science)  

• Tim Donohue

• Lieven Droogmans 

• Mark H. Wood
• Giuseppe Digilio (4Science) 
• Ben Bosman
• Chris Wilper
• Paulo Graça 

7.0 Release Goals

These resources define the prioritization and general schedule we are working towards

• DSpace 7 Release Goals : overview of goals/timelines & beta release process
• DSpace 7 Release Plan spreadsheet:  our planning spreadsheet which details which features are scheduled for each Beta release.

Current Work

Claim a Ticket!

If you do not have access in JIRA or GitHub to officially claim the ticket you wish to work on, contact Tim Donohue

• Available Angular UI Tickets
  
  • Open issues in 7.0beta2 milestone (highest priority)
  • Open issues in 7.0beta3 milestone (second highest priority)
  • Open issues flagged as "Difficulty: low".  These may be a good starting point for newer developers.
• Available REST API Tickets
  
  • Unassigned tickets scheduled for 7.0beta2 (highest priority)
  • Unassigned tickets scheduled for 7.0beta3 (second highest priority)
  • General REST API tickets flagged as "low hanging fruit".  These may be a good starting point for newer developers.
  • General REST API unassigned bug tickets.  Again, might be a good starting point for newer developers.

PRs Needing Review

1. (REST Contract) (low priority) related to the scripts & processes PR above (related to PR 2648 below) https://github.com/DSpace/Rest7Contract/pull/99 (NEEDS UPDATES FROM Kevin Van de Velde (Atmire) ) (Andrea Bollini (4Science) -  REVIEWED, Tim Donohue )
2. (REST Contract) Language support on the backend (possible new cookie for Angular?) https://github.com/DSpace/Rest7Contract/pull/122 (WAITING ON Andrea Bollini (4Science) ) (Tim Donohue, Ben Bosman, Art Lowel (Atmire) )
3. (REST) (low priority) (beta4) Scripts & processes: importing and exporting csv's https://github.com/DSpace/DSpace/pull/2648 (Andrea Bollini (4Science) - REREVIEW, Tim Donohue - REREVIEW  )
4. (REST) (tentative 7.1) [DS-4281]: Metadata suggestions in the live import https://github.com/DSpace/DSpace/pull/2712 (NEEDS TICKET / DISCUSSION) (Tim Donohue -  I believe this is a new feature? We should add this to the spreadsheet, Andrea Bollini (4Science), agree with Tim it is a new feature so postpone compared to existing features)
5. (REST) (beta 3) Subresources should obey access restrictions https://github.com/DSpace/DSpace/pull/2726 (Tim Donohue  -New feedback added, Andrea Bollini (4Science)  implementation approach need discussion, Craig Rosenbeck - Reviewed and added feedback)
6. (REST) Configurable whitelist for "Access-Control-Allow-Origin" header: https://github.com/DSpace/DSpace/pull/2735 (WAITING ON Tim Donohue ) (Ben Bosman , Giuseppe Digilio (4Science))
7. (REST) (beta 3) Account profile management https://github.com/DSpace/DSpace/pull/2747 (Tim Donohue - REREVIEW BY MAY 14 Andrea Bollini (4Science) - reviewed, looks ok suggested some cleanup)
8. (REST) Assume login feature https://github.com/DSpace/DSpace/pull/2740 (Ben Bosman, Tim Donohue - REREVIEW BY MAY 14,  Andrea Bollini (4Science) - reviewed, looks ok suggested some cleanup)
9. (REST) (beta 3) Controlled vocabularyMykhaylo Boychuk https://github.com/DSpace/DSpace/pull/2743 (REST Contract #120) (Tim Donohue, Kevin Van de Velde (Atmire) )
10. (Angular) (beta 3) Edit resource policies https://github.com/DSpace/dspace-angular/pull/645 (Tim Donohue, Art Lowel (Atmire) -   Feedback added,  Julian Timal (eScire) )
11. (Angular) (beta 3) Administer Workflow https://github.com/DSpace/dspace-angular/pull/650 (Tim Donohue, Julian Timal (eScire) - )
12. (Angular) (low priority) (beta4) Scripts & Processes Admin UI https://github.com/DSpace/dspace-angular/pull/636 (Tim Donohue, Giuseppe Digilio (4Science), Craig Rosenbeck)
    1. Depends on REST PR #2648 (see above) ON HOLD
13.  (Angular) (EARLY beta3) Switch to Angular CLI https://github.com/DSpace/dspace-angular/pull/625 ( Tim Donohue, Giuseppe Digilio (4Science)REREVIEW BY MAY 14)
14. (Angular) Alternative links https://github.com/DSpace/dspace-angular/pull/652 (Giuseppe Digilio (4Science) REVIEW BY MAY 14, Tim Donohue - REVIEW BY MAY 14)
15. (Angular) Login as EPerson  https://github.com/DSpace/dspace-angular/issues/653 (Tim Donohue - REREVIEW BY MAY 14, Giuseppe Digilio (4Science) - REVIEW BY MAY 14 ,Julian Timal (eScire))
    
    1. Depends on REST PR #2740 (see above)
16. (Backend) (low priority) DS-626 : Exchange usage data with IRUS https://github.com/DSpace/DSpace/pull/2664 (Craig Rosenbeck, Tim Donohue -  Added feedback)
    
17. (Backend) (low priority) (tentative 7.2) DS-4440 GDPR - Anonymize Statistics Feature: https://github.com/DSpace/DSpace/pull/2692 (Andrea Bollini (4Science), Ben Bosman, Tim Donohue)
18. (Backend) [DS-4149] porting XOAI additional indexer https://github.com/DSpace/DSpace/pull/2756 (Paulo Graça, Andrea Bollini (4Science) )

PRs Coming Soon / On Hold

1. (beta 3) REST Language Support on the backend Mykhaylo Boychuk ETA 17 or 20 April
2. (PR from Kevin coming?) Support Community or Collection Admins adding/managing Community/Collection Groups. See  and this early brainstorm REST Contract PR

PRs Merged this week!

1.   (Backend / Security) Upgrade Spring Boot, Spring & Spring HATEOAS: https://github.com/DSpace/DSpace/pull/2720
2.  (REST) (beta 3) Administer Workflow (Abort WorkflowItem, Delete WorkflowItem) https://github.com/DSpace/DSpace/pull/2727

Blocked

1. (Blocked PRs go here)

Delayed / Needs Discussion

1. Initial Performance Testing from Chris.  Needs revisiting / retesting prior to 7.0. 
   1. https://cwilper.github.io/dspace-perftest/
   2. These performance tests were run prior to the work on "projections" (to limit the data returned by the REST API).  Therefore, it is likely performance is much improved, but needs verification testing.
2. (REST Contract) Edit Homepage News: https://github.com/DSpace/Rest7Contract/pull/45
   
   1. Delayed. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior.  Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata.  Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.  
   2. Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question.  May allow for more CMS-lite behavior in the future
   3. Timeline for this is uncertain.  Possibly in 7 or 8. May depend on how/whether it can be scoped.

Notes

• Discussion Topic #1: Transitioning from BTE to Live Import
  • Decision and reasons behind it are documented in .  We will remove BTE and transition to Live Import.
  • Tim notes that Ben shared an idea to help transition from BTE to Live Import by creating a Live Import "plugin" for BTE.  This allows us to build all tools using Live Import, but temporarily support BTE plugins as we move to migrate them.  Tim notes this is only temporary though. End goal is one framework still.
  • Andrea notes little movement in Live Import since BTE was deprecated.  Tim agreeds, but notes that it is evidence of how complicated it is for us to maintain two frameworks. We tend to not do that well.
  • Also noted that we could adopt BTE and fix security issues.  Tim notes we haven't had a good track record of doing that. See XOAI as an example. We've done little to help that project along in any way, and still haven't even upgraded DSpace to latest version of XOAI.
  • Pascal notes that we should have a plan detailing step by step which plugins will be migrated from BTE to Live Import, when.  When does final removal occur.  Tim agrees, this is not detailed out quite yet and we should do so.
  • ACTION: All agree on a separate meeting to discuss in more detail.  Tim created a Doodle poll for next week: https://doodle.com/poll/sbavqcudm2kzt29t
• Discussion Topic #2: Moving along Subresource access restrictions
  
  • Tim added feedback that Andrea & Craig agree with: https://github.com/DSpace/DSpace/pull/2726#issuecomment-622581818
  • Ideally, first improvement in that feedback would occur in the PR, to ensure some default permissions set across all new endpoints
    • ACTION: Ben will look at this and update PR
  • Second improvement (looking at "PreAuthorize" annotations for exact rights instead of assuming READ privileges) could be in a separate PR
    • If separate, a ticket needs to be created to track this work. It would also require an estimate and be scheduled for a future beta (possibly beta 4?)
    • ACTION: Ben will analyze this and get back to us.
• Discussion Topic #3: Controlled Vocabularies and Authority Control in DSpace 7
  • Summary is on that wiki page.
  • All agree summary is accurate. 
  • Andrea feels that the side effects could be fixed.  Might be able to simply not  store an "Authority ID" for value-pairs or controlled vocabulary.  So, they still may be wrapped in Authority control, but would not see the side effects of having an Authority ID.
  • Tim asks Andrea to document a proposal on how to address these side effects & what the estimate might look like.
    • ACTION: Andrea to create a proposal / estimate for resolving these, ideally shared by Tuesday end of day.
  • Andrea notes obviously if we decide to revert to DSpace 6 behavior we'd also need to estimate that & come up with a proposal
    • Tim agrees, but one step at a time.  Leans towards keeping the new behavior if we can address the side effects. But, if not, we will need to figure out a route towards reverting to DSpace 6 behaviors.