Page History

...

More connection options available at DSpace Meeting Room

Info

title	Planning Sprint #2 : April 20-April 24May 4-8

Catch up on planning / discussion
Review DSpace 7 Release Plan spreadsheet for beta3 and beta4 task

...

(15 mins) Developer Stand Up - Developers give brief updates on their effort (or their team's effort).
- Update/see "Current Work" section below based on your status. Please feel free to update prior to meeting.
- Please highlight any new work (needing reviews/testing), any blockers (for you), and any discussion topics you may have.
(25 30 mins) General Discussion Topics
1. (REST) (beta 3) Subresources should obey access restrictions https://github.com/DSpace/DSpace/pull/2726 Restricted endpoints are sometimes the only HAL link path to public endpoints (10 mins) Tim summarizes next steps for BTE vs Live Import
  Jira
  server DuraSpace JIRA
  Jira
  server DuraSpace JIRA
  serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
  key DS-4495
  )
  1. Some endpoints don't allow you to GET them, but do have publicly accessible child endpoints (e.g. registrations, resourcepolicies, …). To get to those child endpoints the UI can't discover them because the parent endpoint returns a 4xx status code. We should consider a way to prevent a regular response but allow retrieving the HAL links
2. (WAITING ON PROPOSAL FROM 4SCIENCE BEFORE CAN BE DISCUSSED) Revisiting "Collection dropdown in submission" issues (how should we address these? Beta 3 has several other submission tickets, should we include work on these?)
  1. Known performance issues: https://github.com/DSpace/dspace-angular/issues/487 (NOTE: These are still reproducible on the demo site. If you load a new submission, there's a 4-5 second pause before the page becomes usable)
  2. Changing Collection doesn't work when new collection has a different form definition: https://github.com/DSpace/dspace-angular/issues/621
    1. If I recall correctly, Collection item templates also are not working right in this scenario.
    2. And I don't recall whether previously entered metadata gets saved or cleared out when collections have different metadata requirements.
    3. Since it's possible to start a new submission via a default Collection, this increases the likelihood of encountering these issues (if the deposit starts in the wrong collection).
  3. somehow related to this discussion Creation of DSpace Entities
3. (WAITING ON FEEDBACK FROM ATMIRE BEFORE CAN BE DISCUSSED)Finalize / approve the initial list of all authorization features which we should implement for the /api/authz/features REST endpoint. This list of features should be limited to only features which are required to enable/disable User Interface functionality. (In other words, we can always add more features in the future. We just need to approve the list necessary for 7.0)
  1. Review current spreadsheet (from Andrea Bollini (4Science) ) : https://docs.google.com/spreadsheets/d/1182LcD_WqIZRbUGWpLtBw0aOMR9jhbOVB7GZqtTpR9A/edit?usp=sharing
4. 4491
  1. BTE framework was deprecated in 6.0, with Live Import framework selected as the replacement.
  2. As discussed previously, the "Metadata Suggestions" PR (https://github.com/DSpace/DSpace/pull/2712) includes useful enhancements to Live Import functionality (from how it existed in 6.x). However, this doesn't quite replace existing BTE functionality.
  3. Tim Donohue's recommendation is to start from a PR that implements a more simplistic implementation of Live Import that can be used to swap out BTE more directly. Would this be possible?
    1. The idea would be to start small here with a basic PR that disables/replaces BTE from REST API in favor of Live Import. The focus would be on replacing/refactoring the two `dspace-server-webapp` classes that touch BTE as documented in this JIRA comment
    2. We would then follow that up with additional PRs to add/enhance functionality of Live Import to better align with (other) existing features in BTE. Including PR#2712, but also including support for additional formats/services and perhaps improved command-line support (if deemed necessary)
  4. Ben Bosman has suggested that we might consider creating a BTE "plugin" for Live Import to help with the transition. This will allow us to build all DSpace 7 features using the Live Import framework, while (temporarily) allowing BTE features/functions to still be able to work via Live Import. (Tim notes that this should only be temporary to avoid short term feature loss. The end goal should still be one framework.)
  5. If others feel necessary, a separate meeting specific to BTE & Live Import could be scheduled to talk through the details of this change.
5. (10 mins) (REST) (beta 3) Subresources should obey access restrictions https://github.com/DSpace/DSpace/pull/2726
  1. Tim Donohue added feedback to the PR based on testing/analysis of Spring Security annotations: https://github.com/DSpace/DSpace/pull/2726#issuecomment-622581818
6. (10 mins) How to move forward on Controlled Vocabulary feature: https://github.com/DSpace/DSpace/pull/2743
  1. Ben Bosman added feedback on feature gaps between DSpace 6 and DSpace 7 with regards to Controlled Vocabulary. How can we work to address these gaps in upcoming PR(s)?
  2. Early notes on differences in behavior: Controlled Vocabularies and Authority Control in DSpace 7
  3. This topic may be short on time, however if someone(s) is willing to draft a proposal for addressing these gaps, we can revisit next week.
7. Tabled Topics (will not be discussed unless time allows)
  1. Restricted endpoints are sometimes the only HAL link path to public endpoints (
  Tabled Topics (two were tabled for future discussion)
  1. (PR from Kevin coming?) We do not fully support Community or Collection Admins adding/managing Community/Collection GroupsRelated to
    Jira
    server DuraSpace JIRA
    serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
    key DS-
    4486 and this brainstorm REST Contract PR
  2. Currently, Community/Collection Admins do not have permissions to search/view all EPeople or Groups, nor can they view current members of the Group. These are all limited to full Administrators at this time.
  3. As discussed on April 16, the ideal solution would be to not change the REST API contract, but instead allow all Community/Collection Admins to have search/view permissions on all EPeople or Groups. However, this would require a way to easily determine if a user was a Community/Collection Admin in a manner that does not cause performance problems. For example, we would not want to have to load every Group or every Community/Collection to determine if the current user could manage/administer them. Instead, we'd want to see if there's a database level query that can answer the question: "Does the current user have Admin-level rights on one or more Communities or Collections?"
  4. (On hold for beta5 and then 7.1) Live Import Feature & "Metadata Suggetions" per https://github.com/DSpace/DSpace/pull/2712
    1. Live Import was definitely in DSpace 6.x: 2016 Framework for live import from external sources
    2. However, "Metadata Suggestions" seems like a new feature of Live Import? If this is accurate, we should track it separately so that it can have estimates & scheduling specific to it.
8. (Feel free to add other topics)
(20 mins) Planning for next week
- Claim tasks for next week from DSpace 7 Release Plan spreadsheet
- Assigning PRs for Review

Attendees

7.0 Release Goals

These resources define the prioritization and general schedule we are working towards

DSpace 7 Release Goals : overview of goals/timelines & beta release process
DSpace 7 Release Plan spreadsheet: our planning spreadsheet which details which features are scheduled for each Beta release.

Current Work

...

title	Legend for status icons

= Highest Priority tasks (please prioritize these reviews/tasks over others).

= review done, changes were requested or bugs found.

= review done, approved.

= review done, merge conflict or other minor changes requests

1. 1. 4495
    )
    1. Some endpoints don't allow you to GET them, but do have publicly accessible child endpoints (e.g. registrations, resourcepolicies, …). To get to those child endpoints the UI can't discover them because the parent endpoint returns a 4xx status code. We should consider a way to prevent a regular response but allow retrieving the HAL links
  2. Finalize / approve the initial list of all authorization features which we should implement for the /api/authz/features REST endpoint. This list of features should be limited to only features which are required to enable/disable User Interface functionality. (In other words, we can always add more features in the future. We just need to approve the list necessary for 7.0)
    1. Review current spreadsheet (from Andrea Bollini (4Science) ) : https://docs.google.com/spreadsheets/d/1182LcD_WqIZRbUGWpLtBw0aOMR9jhbOVB7GZqtTpR9A/edit?usp=sharing
      1. Art Lowel (Atmire) : I don't see any immediate issues with the current set of features, but I would prefer a consistent naming scheme. I'd use canDoSomething for everything
  3. PROPOSAL FROM 4SCIENCE https://wiki.lyrasis.org/x/jwUoCw
    1. Revisiting "Collection dropdown in submission" issues (how should we address these? Beta 3 has several other submission tickets, should we include work on these?)
    2. Known performance issues: https://github.com/DSpace/dspace-angular/issues/487 (NOTE: These are still reproducible on the demo site. If you load a new submission, there's a 4-5 second pause before the page becomes usable)
    3. Changing Collection doesn't work when new collection has a different form definition: https://github.com/DSpace/dspace-angular/issues/621
      1. If I recall correctly, Collection item templates also are not working right in this scenario.
      2. And I don't recall whether previously entered metadata gets saved or cleared out when collections have different metadata requirements.
      3. Since it's possible to start a new submission via a default Collection, this increases the likelihood of encountering these issues (if the deposit starts in the wrong collection).
    4. somehow related to this discussion Creation of DSpace Entities
(15 mins) Planning for next week
- Claim tasks for next week from DSpace 7 Release Plan spreadsheet
- Assigning PRs for Review

Attendees

7.0 Release Goals

These resources define the prioritization and general schedule we are working towards

DSpace 7 Release Goals : overview of goals/timelines & beta release process
DSpace 7 Release Plan spreadsheet: our planning spreadsheet which details which features are scheduled for each Beta release.

Current Work

Panel

title	Legend for status icons

= Highest Priority tasks (please prioritize these reviews/tasks over others).

= review done, changes were requested or bugs found.

= review done, approved.

= review done, merge conflict or other minor changes requests

Status


colour	Blue
title	1 approval

= pull request only requires a single approval to merge. This is generally reserved for PRs which are either smaller, obvious, and/or bug fixes with tests to prove they work.

Claim a Ticket!

If you do not have access in JIRA or GitHub to officially claim the ticket you wish to work on, contact Tim Donohue

Available Angular UI Tickets
- Open issues in 7.0beta2 milestone (highest priority)
- Open issues in 7.0beta3 milestone (second highest priority)
- Open issues flagged as "Difficulty: low". These may be a good starting point for newer developers.
Available REST API Tickets
- Unassigned tickets scheduled for 7.0beta2 (highest priority)
- Unassigned tickets scheduled for 7.0beta3 (second highest priority)
- General REST API tickets flagged as "low hanging fruit". These may be a good starting point for newer developers.
- General REST API unassigned bug tickets. Again, might be a good starting point for newer developers.

PRs Needing Review

(REST Contract) (low priority) related to the scripts & processes PR above (related to PR 2648 below) https://github.com/DSpace/Rest7Contract/pull/99 (NEEDS

...

Claim a Ticket!

If you do not have access in JIRA or GitHub to officially claim the ticket you wish to work on, contact Tim Donohue

Available Angular UI Tickets
- Open issues in 7.0beta2 milestone (highest priority)
- Open issues in 7.0beta3 milestone (second highest priority)
- Open issues flagged as "Difficulty: low". These may be a good starting point for newer developers.
Available REST API Tickets
- Unassigned tickets scheduled for 7.0beta2 (highest priority)
- Unassigned tickets scheduled for 7.0beta3 (second highest priority)
- General REST API tickets flagged as "low hanging fruit". These may be a good starting point for newer developers.
- General REST API unassigned bug tickets. Again, might be a good starting point for newer developers.

PRs Needing Review

(REST Contract) (low priority) related to the scripts & processes PR above (related to PR 2648 below) https://github.com/DSpace/Rest7Contract/pull/99 (NEEDS UPDATES FROM Kevin Van de Velde (Atmire) ) (Andrea Bollini (4Science) - REVIEWED, Tim Donohue )
(REST Contract) Language support on the backend (possible new cookie for Angular?) https://github.com/DSpace/Rest7Contract/pull/122 (WAITING ON Andrea Bollini (4Science) ) (Tim Donohue, Ben Bosman, Art Lowel (Atmire) )
(REST) (low priority) (beta4) Scripts & processes: importing and exporting csv's https://github.com/DSpace/DSpace/pull/2648 (WAITING ON PR UPDATES FROM Kevin Van de Velde (Atmire) ) (Andrea Bollini (4Science) - REVIEWED, Tim Donohue - added summary of way forward Mark H. Wood )
(REST ) (tentative 7.1) [DS-4281]: Metadata suggestions in the live import Contract) Language support on the backend (possible new cookie for Angular?) https://github.com/DSpace/DSpaceRest7Contract/pull/2712122 (NEEDS TICKET / DISCUSSION(WAITING ON Andrea Bollini (4Science) ) (Tim Donohue - I believe this is a new feature? We should add this to the spreadsheet, Andrea Bollini (4Science), agree with Tim it is a new feature so postpone compared to existing features, Ben Bosman, Art Lowel (Atmire) )
(REST) (low priority) (beta 3) Subresources should obey access restrictions beta4) Scripts & processes: importing and exporting csv's https://github.com/DSpace/DSpace/pull/27262648 (Tim Donohue -New feedback added, Andrea Andrea Bollini (4Science) implementation approach need discussion, Craig Rosenbeck - REREVIEW, Tim Donohue - REREVIEW )
(REST) (beta 3) Administer Workflow (Abort WorkflowItem, Delete WorkflowItem) tentative 7.1) [DS-4281]: Metadata suggestions in the live import https://github.com/DSpace/DSpace/pull/27272712 (Tim NEEDS TICKET / DISCUSSION) (Tim Donohue - REREVIEW I believe this is a new feature? We should add this to the spreadsheet, Andrea Bollini (4Science) REREVIEW TODAY, agree with Tim it is a new feature so postpone compared to existing features)
(REST) Configurable whitelist for "Access-Control-Allow-Origin" header: (beta 3) Subresources should obey access restrictions https://github.com/DSpace/DSpace/pull/27352726 (WAITING ON Tim Donohue) (Ben Bosman , Giuseppe Digilio (4Science))(REST) (beta 3) Account profile management -New feedback added, Andrea Bollini (4Science) implementation approach need discussion, Craig Rosenbeck - Reviewed and added feedback)
(REST) Configurable whitelist for "Access-Control-Allow-Origin" header: https:https://github.com/DSpace/DSpace/pull/27472735 (WAITING ON Tim Donohue - REVIEW BY MAY 7, Andrea Bollini ) (Ben Bosman , Giuseppe Digilio (4Science) - REVIEW BY MAY 7)
(REST) Assume login feature ) (beta 3) Account profile management https://github.com/DSpace/DSpace/pull/27402747 (Ben Bosman, Tim Donohue - REVIEW REREVIEW BY MAY 7,14 Andrea Bollini (4Science) - REVIEW BY MAY 7reviewed, looks ok suggested some cleanup)
(REST) (beta 3) Controlled vocabulary (TALK NEXT WEEK ONCE WE HAVE A BUG TICKET FROM ATMIRE) Mykhaylo Boychuk Assume login feature https://github.com/DSpace/DSpace/pull/27432740 (REST Contract #120) (Ben Bosman, Tim Donohue - REREVIEW BY MAY 14, Andrea Bollini (4Science) - reviewed, looks ok suggested some cleanup)
(REST) (beta 3) Controlled vocabularyMykhaylo Boychuk https://github.com/DSpace/DSpace/pull/2743 (REST Contract #120) (Tim Donohue, Kevin Van de Velde (Atmire) ), Kevin Van de Velde (Atmire) )
(Angular) (beta 3) Edit resource policies https://github.com/DSpace/dspace-angular/pull/645 (Tim Donohue, Art Lowel (Atmire) - Feedback added, Julian Timal (eScire) )
(Angular) (beta 3) Administer Workflow https://github.com/DSpace/dspace-angular/pull/650 (Tim Donohue, Julian Timal (eScire) - )
Depends on REST PR #2727 (see above).

(

(low priority)

https://github.com/DSpace/dspace-angular/pull/636

Tim Donohue

Giuseppe Digilio (4Science)

Craig Rosenbeck

Depends on REST PR #2648 (see above) ON HOLD

(Angular) (EARLY beta3) Switch to Angular CLI https://github.com/DSpace/dspace-angular/pull/625 ( Tim Donohue - REVIEW BY MAY 7, Giuseppe Digilio (4Science)REREVIEW BY MAY 714)
(Angular) Alternative links https://github.com/DSpace/dspace-angular/pull/652 (Giuseppe Digilio (4Science) REVIEW BY MAY 14, Tim Donohue - REVIEW BY MAY 14)
(Angular) Login as EPerson https://github.com/DSpace/dspace-angular/issues/653 (Tim Donohue - REREVIEW BY MAY 714, Giuseppe Digilio (4Science) - REVIEW BY MAY 714 ,Julian Timal (eScire))
1. Depends on REST PR #2740 (see above)
(Backend) (low priority) DS-626 : Exchange usage data with IRUS https://github.com/DSpace/DSpace/pull/2664 (Craig Rosenbeck, NEEDS SECOND REVIEWERTim Donohue - Added feedback)
(Backend) (low priority) (tentative 7.2) DS-4440 GDPR - Anonymize Statistics Feature: https://github.com/DSpace/DSpace/pull/2692 (Andrea Bollini (4Science), Ben Bosman, Tim Donohue)
(Backend / Security) (EARLY beta3) Upgrade Spring Boot, Spring & Spring HATEOAS: ) [DS-4149] porting XOAI additional indexer https://github.com/DSpace/DSpace/pull/27202756 (Paulo Graça, Andrea Bollini (4Science) , Craig Rosenbeck, Kevin Van de Velde (Atmire) )

PRs Coming Soon / On Hold

(beta 3) REST Language Support on the backend Mykhaylo Boychuk ETA 17 or 20 April

PRs Merged this week!

(first PR merged goes here)

Blocked

(Blocked PRs go here)

Delayed / Needs Discussion

...

In July 25 meeting, we noted this probably cannot be resolved with just one simple solution. May need to look at different options for different scenarios
Work is ongoing, but has been started in these areas:
1. Summary of ideas: REST Authorization
2. Contract for Authorization Endpoints: https://github.com/DSpace/Rest7Contract/pull/92
3. Contract for ResourcePolicies: https://github.com/DSpace/Rest7Contract/pull/87

...

https://cwilper.github.io/dspace-perftest/

...

PR from Kevin coming?) Support Community or Collection Admins adding/managing Community/Collection Groups. See
Jira
server DuraSpace JIRA
serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
key DS-4486
and this early brainstorm REST Contract PR

PRs Merged this week!

(Backend / Security) Upgrade Spring Boot, Spring & Spring HATEOAS: https://github.com/DSpace/DSpace/pull/2720
(REST) (beta 3) Administer Workflow (Abort WorkflowItem, Delete WorkflowItem) https://github.com/DSpace/DSpace/pull/2727

Blocked

(Blocked PRs go here)

Delayed / Needs Discussion

Initial Performance Testing from Chris. Needs revisiting / retesting prior to 7.0.
1. https://cwilper.github.io/dspace-perftest/
2. These performance tests were run prior to the work on "projections" (to limit the data returned by the REST API). Therefore, it is likely performance is much improved, but needs verification testing.
(REST Contract) Edit Homepage News: https://github.com/DSpace/Rest7Contract/pull/45
1. Delayed. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior. Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata. Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.
2. Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question. May allow for more CMS-lite behavior in the future
3. Timeline for this is uncertain. Possibly in 7 or 8. May depend on how/whether it can be scoped.

Notes

Discussion Topic #1: Transitioning from BTE to Live Import
- Decision and reasons behind it are documented in
  Jira
  server DuraSpace JIRA
  serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
  key DS-4491
  . We will remove BTE and transition to Live Import.
- Tim notes that Ben shared an idea to help transition from BTE to Live Import by creating a Live Import "plugin" for BTE. This allows us to build all tools using Live Import, but temporarily support BTE plugins as we move to migrate them. Tim notes this is only temporary though. End goal is one framework still.
- Andrea notes little movement in Live Import since BTE was deprecated. Tim agreeds, but notes that it is evidence of how complicated it is for us to maintain two frameworks. We tend to not do that well.
- Also noted that we could adopt BTE and fix security issues. Tim notes we haven't had a good track record of doing that. See XOAI as an example. We've done little to help that project along in any way, and still haven't even upgraded DSpace to latest version of XOAI.
- Pascal notes that we should have a plan detailing step by step which plugins will be migrated from BTE to Live Import, when. When does final removal occur. Tim agrees, this is not detailed out quite yet and we should do so.
- ACTION: All agree on a separate meeting to discuss in more detail. Tim created a Doodle poll for next week: https://doodle.com/poll/sbavqcudm2kzt29t
Discussion Topic #2: Moving along Subresource access restrictions
- Tim added feedback that Andrea & Craig agree with: https://github.com/DSpace/

...

- DSpace/pull/

...

Delayed. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior. Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata. Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.
Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question. May allow for more CMS-lite behavior in the future
Timeline for this is uncertain. Possibly in 7 or 8. May depend on how/whether it can be scoped.

- 2726#issuecomment-622581818
- Ideally, first improvement in that feedback would occur in the PR, to ensure some default permissions set across all new endpoints
  - ACTION: Ben will look at this and update PR
- Second improvement (looking at "PreAuthorize" annotations for exact rights instead of assuming READ privileges) could be in a separate PR
  - If separate, a ticket needs to be created to track this work. It would also require an estimate and be scheduled for a future beta (possibly beta 4?)
  - ACTION: Ben will analyze this and get back to us.
Discussion Topic #3: Controlled Vocabularies and Authority Control in DSpace 7
- Summary is on that wiki page.
- All agree summary is accurate.
- Andrea feels that the side effects could be fixed. Might be able to simply not store an "Authority ID" for value-pairs or controlled vocabulary. So, they still may be wrapped in Authority control, but would not see the side effects of having an Authority ID.
- Tim asks Andrea to document a proposal on how to address these side effects & what the estimate might look like.
  - ACTION: Andrea to create a proposal / estimate for resolving these, ideally shared by Tuesday end of day.
- Andrea notes obviously if we decide to revert to DSpace 6 behavior we'd also need to estimate that & come up with a proposal
  - Tim agrees, but one step at a time. Leans towards keeping the new behavior if we can address the side effects. But, if not, we will need to figure out a route towards reverting to DSpace 6 behaviors.

...

We've decided (in meeting on March 7, 2019) to use ETags to implement concurrency. REST Contract notes on ETags: https://github.com/DSpace/Rest7Contract#etags--conditional-headers
ETags only update of the two fields match. If someone edits first, your edit would fail and you would get a fail response (422?)
ETags seems to have broader support in other REST APIs. Recommended also by both Art and Andrea.

...

Page tree

Versions Compared

Old Version 1

New Version Current

Key

Attendees

7.0 Release Goals

Current Work

Attendees

7.0 Release Goals

Current Work

Claim a Ticket!

PRs Needing Review

Claim a Ticket!

PRs Needing Review

PRs Coming Soon / On Hold

PRs Merged this week!

Blocked

Delayed / Needs Discussion

PRs Merged this week!

Blocked

Delayed / Needs Discussion

Notes