Page History

...

• Lots of individuals just returning from holidays.  Welcome back!
• Quick update on Improved Estimation Strategy for Beta release.
  • Heather Greer Klein has been working on this while Tim was out on holiday. She has some initial spreadsheets nearly ready, and Tim will be reviewing today/tomorrow
  • Anticipate that estimation spreadsheets will be ready to share either on Aug 29 or Sept 5....once ready, we'll go through them in detail and get some estimators lined up, etc.
• Discussion: Revisiting REST API Projections.
  • Relevant tickets: 

    • Jira
      server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-3533

    • Jira
      server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-4306

    • Jira
      server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-4307

  • Main point is to improve performance by limiting the amount of information returned via the REST API.
    • Performance will improve because REST API is gathering less information (via DB & Hibernate), and UI is retrieving/parsing less information (smaller JSON)
  • Old PR (from Tom Desair, previously of Atmire) began some Projections work based on Spring Data REST Projections
    • https://github.com/DSpace/DSpace/pull/1847
    • Basic idea is that Spring Data REST provides a `@Projections` annotation that can be used to define named projections.  In requests to the REST API, by default you still get all data back, but you can pass in a "projections=" parameter to limit the information you need based on one (or more) defined projections.
  • Will require changes both to REST API and Angular UI
    • Art Lowel (Atmire) notes that UI's current caching system may need some rethinking.  All objects currently cached in UI layer.  If some requests are only returning partial objects (e.g. metadata only), then the UI may need to "know" when it only has a partial object – so that it can make a new request for the full object when necessary.  Ideally though, we obviously should figure out the REST API implementation first, then brainstorm how this affects the UI cache.
    • Andrea Bollini (4Science) notes that Tom's initial PR had a good approach of building off Spring Data REST Projections... however , as noted in this PR comment, Andrea feels that the Projection itself should be using the "*Rest" classes, like ItemRest.class, (instead of the Hibernate classes, like Item.class), as Projections are a REST API level concept...and should therefore be implemented at that level.  Currently, it seems odd that they are bypassing the "*Rest" classes to define projections on the Hibernate classes themselves
      • Tim notes point well taken. However, it sounds like this needs further investigation / thinking on how to change the implementation to use "*Rest" classes. Perhaps we can do this in two stages?  Start with Tom's current strategy just to get an initial implementation (that the UI can start to use & implement).  Then, we can refactor to use the "*Rest" Classes once we figure out a way to do so.
      • Andrea agrees.  We should do the REST Contract first, then an initial implementation to build from.  Will need to scope the idea of Projections to make this easier to approach.
  • Scoping Projections: What types of Projections do we need?
    • Andrea notes that Projections should be based on use cases, e.g. a "list" of objects (Items, Collections, Communities) should return less information about objects in that list than an Object page would return.
    • Tim suggests we start small.  One initial projection.
    • ACTION: Start with a "list" projection that ONLY returns:  all metadata and one thumbnail (if exists)
      • Others agree.  Andrea notes this also seems like it would resolve our two outstanding bugs related to projections (DS-4306 and DS-4307, see above for links)
  • Next Steps:
    • Need a REST Contract for Projections. This is already somewhat detailed in Tom's initial PR's description...just needs to be formalized into a REST Contract
    • Initial implementation will be limited to a projection(s) for the "list" use case (all metadata + thumbnail).  This might be one "list" projection, or maybe two projections: "metadata only" projection and "thumbnail" projection.
    • Keep an eye out for other possible Projection use cases / needs...this "list" projection though seems to be the biggest need (and may provide massive performance improvements)
    • May then need to investigate using the "*Rest" classes & refactor that initial implementation (see comments from Andrea above)
  • Who will start this work?
    • Ben notes that Chris Wilper might be interested.  Will check with him though
    • ACTION: Tim Donohue will update JIRA tickets with what we decided today & then we can look to assign the ticket(s).
• Assigning PR Reviews
  • One discussion topic came up
  • Questions on Bitstream Sequence ID in two PRs: https://github.com/DSpace/Rest7Contract/pull/68 and https://github.com/DSpace/DSpace/pull/2473
  • Should Sequence ID still exist in DSpace 7?  It seemed to be used as an alternative URL for bitstream downloads (in DSpace 6).  Bitstreams could be downloaded via sequence ID (unique in an Item) or name, or both.
  • Sequence ID is misnamed. It's not defining a sequence (ordering) in terms of Bitstreams...it's also not managed by the database layer (not a database sequence).  It's an integer defined (by the UI in DSpace 6) for each bitstream, and it must be unique within an Item.
  • Andrea notes that Sequence ID had another purpose in DSpace 6....it was a "persistent" Identifier for Bitstreams, especially since a Bitstream's name could be changed
  • Ben notes that in DSpace 7, the Bitstream's "persistent" Identifier is its UUID
  • We need to keep Sequence IDs around in DSpace 7 to allow for redirects from old (DSpace 6 style) URLs to new URLs
  • However, the way they were managed in DSpace 6 was odd.  In DSpace 7, we'll make Sequence IDs READ-ONLY at the REST API layer.
  • Need a ticket to discuss whether Sequence IDs should remain a key feature (in which case they probably should be auto-assigned by the Database or Java API layer, they currently are not), or if they are only being kept for backwards compatible URLs (redirecting pre-DSpace 7 URLs to new URLs).  Tim Donohue will create this ticket
  • POST-MEETING UPDATE: After further analysis, Tim realized that Sequence IDs are being managed by ItemService.update().  Therefore they should not  be deprecated.  Instead, Tim created a small PR to better document sequence IDs in the code / javadocs: https://github.com/DSpace/DSpace/pull/2492/