...
- Qa Sinopia Collaboration – Support and evolve QA+cache instance for use with Sinopia
- 2021-0607-11
- In User Stories for Sinoipia-QA/cache-ShareVDE - 4 tabs: list of models being supported for this grant period, how to create new data, editing data, and summary of what is work remaining to meet workflows identified and agreed upon. Ideal: API with RDF. Adequate option: SVDE provides downloads and we cache those in Dave's system. Undesired: provide API that is not RDF and we're expected to connect to that. For dereferencing, similar options – ideal is they dereference and we can do via QA or Sinopia can connect directly. Either they provide dereference point or we do. What has to happen at the three levels we support is all up to what SVDE chooses
- No meeting with Stanford this week. Met with ShareVDE to talk about datastores and APIs. See Q&A doc for more info and followup questions. Primary take aways...
- 2 tenants:
- PCC
- Everything else
- 3 datastores:
- Postgres DB
- primary datastore into which data is ingested sychronously
- data includes CKB and (I think) institutional bibliographic data as (I think) BibFrame
- Solr
- derived from Postgres DB asychronously
- only enough data to fulfill full text searching (I'm pretty sure)
- Stardog - derived from Postgres DB asychronously
- data includes CKB and institutional bibliographic data as BibFrame (I'm pretty sure)
- Postgres DB
- 3 APIs
- GraphQL
- search Postgres for keywords using boolean search; search solr for full text search
- returned data is JSON with shape based on query
- REST API
- same datastores searched as GraphQL
- returned data is JSON with predetermined shape based on entity
- SPARQL endpoints (one for PCC, one for everything else)
- search Stardog
- returned data is RDF with shape determined by SPARQL query
- GraphQL
- Data planned to be released in 3 stages with Stage A including Stanford & LOC
- 2 tenants:
- Did an overview of containerization. Hoping to have discussions with SVDE about data sources and APIs. Want to answer both workflow questions (what we will support) and how we can do it (using technologies that are available now)
- 2021-0607-11
- Best Practices for Authoritative Data working group (focus on Change Management)
- 2021-06-11
- Continue to talk about types of change and what data goes with that. Key piece: challenges associated with doing this approach at all; related to the fact that it is linked data. If something more complex like BF where you have to traverse the graph, cannot just say what the new label is (for instance). Three audiences: full cache audience who wants everything. application audience who has only cached small subset of data who want minimal info with limited interaction to update cache. Humans who want focused interactions with change management.
- 2021-06-25
- Discussion shows that there is complexity in every type of change, because in RDF there is a flexible definition of boundary. Looking at the use of RDF*
- 2021-06-11
- Cache Containerization Plan - Develop a sustainable solution that others can deploy
- Consider moving live QA instance from EBS to container version? Need to consider update mechanisms CI/CD. Agree that this is a good direction and Greg/Lynette will discuss
- 2021-06-22
- Lynette received from Dave what she needs to create a container. did not immediately work so needs to dig into it
- 2021-06-25
- Greg made some progress on containerization. Has a Jenkins job that can pull from github and deploy to the running service. This is key to replacing the ElasticBeanstalk version. Can now design the update process and then think about replace the beanstalk version
- Lynette will be working on this next week: firm up instructions for QA servers and getting containers for Dave's work
- Also need to think about LD4 conference presentation, week of July 19
...
- https://github.com/LD4P/discovery/projects/2 for issues etc.
- Draft of a discovery plan: https://docs.google.com/document/d/1zKYW7FQVVNvyd0XjjW0qWznX9PC3jbmOE6Kz_yygPjs/edit?usp=sharing
- Research: how to go from knowledge graph to an index
- DASH! (Displaying Authorities Seamlessly Here)
- Dashboard design meeting kickoff notes
- User reps D&A meeting: Expect next follow-up in August (Slides: from user reps meeting 2021-04-09 and result was "not no")
- https://docs.google.com/document/d/1PgQi3xobsPhr9DUHU_YGeimL1OjNiiTdkiNWb36r3Gg/edit
- Usability testing and followup for DASH: Usability results
- Usability results, a few little things to finish up
- GitHub issues
- How long to continue working on DASH! ?
- 2021-06-11 Tim and Huda had a meeting to discuss priorities and approach. Main concerns were showing user reps advancement in features and . Tim working on mockups on redesign; will not implement – but will present to user reps to decide which options. Not prioritizing anything functional at-present. Will make sure pages work well when there is not enough data on the page. Will put prototype in position that we feel more comfortable when people play with it. Concerned that if show same thing at user reps, will be unproductive... but question whether anyone will remember. With more robust prototype, hope will yield more of a decision. Tim does not have much time to work on this (2.5-3 weeks) – aiming for user rep meeting in August.
- 2021-06-25
- Huda hasn't really worked much on DASH but plans on devoting 1.5 -2 weeks during July. (This should leave the rest for BANG!)
- FOLIO is somewhat consuming right now
- 2021-07-02
- Huda and Tim discussed work to do on DASH before user reps meeting
- Video for DASH!, theme?
- Sonic? Roadrunner?
- Youtube Creative Commons License filter (hope maybe?)
- BANG! (Bibliographic Aspects Newly GUI'd)
- Jamboard link
- Expect to include Works. Need to do something beyond what we already have live from the OCLC concordance data.
- Full OCLC concordance us 343M rows, and gzipped the file is 3.3GB
- SVDE Works
...