Page History

...

• Qa Sinopia Collaboration – Support and evolve QA+cache instance for use with Sinopia
  
  • 2021-05-2128
    • We began discussing the document describing interaction patterns between Sinopia-QA/cache-ShareVDE.  The key take away from the discussion is that there needs to be clarity about how the PCC data and Stanford Institution data are expected to be used.  There are some key questions to be answered that will drive the technological solutions in Sinopia, QA/cache, and ShareVDE.  They include:
      • How is data initially ingested into ShareVDE?
      • Once initial ingest is complete, how is new data added (e.g. incremental ingests from original ingest source, newly created entities in Sinopia)?
      • Where is the Source of Truth (e.g. ShareVDE, Sinopia)?
      • How are updates made to entities?  The short answer is that edits are expected to happen in Sinopia.  But the details of how editing will happen is highly dependent on the answer the Source of Truth question
    • Vivian from Stanford joined this week to get an update on ShareVDE.  She plans to attend regularly to help solidify the expectations of interactions.  As a result, a document was created to make clear the interaction patterns for ShareVDE-QA/cache-Sinopia.  It includes definition of workflows (e.g. field lookup, cloning), shared workflows (e.g. search, dereferencing URIs), ingest workflows, and descriptions of ShareVDE APIs.
      • Agree that we want to get detailed description of specific interactions, want to better define the full circle
    • Dave has completed the ingest of the ShareVDE PCC data.  It is not clear if it is fully indexed yet.  I am waiting for the batch and lookup URLs for the cache service to be able to setup the QA config
      • .
• Best Practices for Authoritative Data working group (focus on Change Management)
  • 2021-05-2128
    
    • Second meeting this past Monday.  We continued discussing type of changes.  No new types were added.  We began the process of discussing what information is needed in the change management stream for each type. 
    • We talked some about NEW entities identifying two options.  We did not make a decision on which approach is preferred.  (NOTE: Format here is to convey data and is not necessarily the final recommended format.}
      • { 'type': 'NEW', 'URI': 'https://uri.for.new.entity' } with that, the downstream consumer can dereference the URI and use the results to add the entity to the cache

      • { 'type': 'NEW', 'URI': 'https://uri.for.new.entity', entity: { json-ld for new entity }  } with that, the downstream consumer can use the entity in the change management stream to add the entity to the cache

    • We spent a good bit of time on CHANGE LABEL which turned out to be more complex than expected.  The purpose of this type is to facilitate applications that cache labels for quick display  in the application or for indexing labels to facilitate search.  Again two options were identified.
      • { 'type': 'CHANGE_LABEL', 'URI': 'https://uri.for.new.entity', 'predicate': 'skos:primaryLabel', 'OLD_LABEL': 'old literal'@en, 'NEW_LABEL': 'new literal'@en } with OLD_LABEL being optional.  Without OLD_LABEL, this is a new label.  With it, the OLD_LABEL is being replaced with the new label.  Applications can search their caches for the OLD_LABEL triple and update it to the NEW_LABEL.
      • { 'type': 'REMOVE_LABEL', 'URI': 'https://uri.for.new.entity', 'predicate': 'skos:primaryLabel', 'LABEL': 'old literal'@en }
        { 'type': 'ADD_LABEL', 'URI': 'https://uri.for.new.entity', 'predicate': 'skos:primaryLabel', 'LABEL': 'new literal'@en }  I have a question about how the application will process the change management stream.  It will need to know that these two change management documents are related.  This is fine for a full cache, but may not help applications that are only caching the labels
      First meeting occurred.  We got through use cases pretty quickly.  Use Cases include Cache of Full Download, Cache of Labels in an Application, Caching label and URIs in MARC records, Locally created authority data. We began discussing type of changes.  At the end, the group was encouraged to identify existing approaches to providing change data
      • .
• Cache Containerization Plan - Develop a sustainable solution that others can deploy
  • 2021-04-30
    • Have worked on permissions issues and documented how to implement in AWS
    • Greg now running out of things to do without more input from Dave. Can document existing work and develop presentation for conference
    • Consider moving live QA instance from EBS to container version? Need to consider update mechanisms CI/CD. Agree that this is a good direction and Greg/Lynette will discuss
  • 2021-05-07
    • Greg spent some time learning about github actions for CI/CD. Has some permissions issue with github repo. Expect to continue working on this today and hope to be able to redeploy container when content changes
  • 2021-05-21
    • Greg and Lynette have discussed work on containerization of DAVE

...