Page History

...

https://github.com/LD4P/discovery/projects/2 for issues etc.
Draft of a discovery plan: https://docs.google.com/document/d/1zKYW7FQVVNvyd0XjjW0qWznX9PC3jbmOE6Kz_yygPjs/edit?usp=sharing
Research: how to go from knowledge graph to an index
- Research decision points, Use cases
DASH! (Displaying Authorities Seamlessly Here)
- 2022-01-28 02-11 - Documentation is DONE
  - See WP3: Discovery
  - Only things left are formatting of writeup and finishing off documentation. Aiming for next week
BANG! (Bibliographic Aspects Newly GUI'd)
- Jamboard link
- Expect to include Works. Need to do something beyond what we already have live from the OCLC concordance data.2021-12-10To dos: Still need to use the work to work relationships data (which is based on a partial query run) (i.e. related, etc.) and bring that to the page. Will try to get that done by end of year. Using Wikidata relationships between item and IMDB to get appropriate IMDB link
- Design work/review options for displaying information to users (works/instances and how/whether to modify the current catalog view focus on item)
- 2022-01-21
  - Starting data source writeup
    - Overview tries to get at counts for information.
    - Started some diagrams
    - Deep thought seems to perform better for some of these queries so used that for some of the counts
  - Decided to take work to work relationship ISBNs to see which pairs had matches in our catalog (i.e. work1 and work2 have a relationship, work1 has an instance i1, work2 has an instance i2, result = ISBNs for i1 and i2). Saw overlap between ISBNs from different works and tracking that down with Steven.
  - Also: ran work to work relationship query again (where first work has instance with ISBN) to get complete result. Used this approach because trying to run a full query was taking forever. Second script tries to get instances with ISBNs for second works to create two groups of ISBNs and the predicate relating the two works
    - 33,395 lines in result (i.e. potential sets of related ISBNs)
    - But: Need to review to see how many of these include the same ISBN on both sides of the relationship (i.e. run yet another script)
    - And: How many of these matching groups will have catalog results
  - Will also run comparable queries on Stanford CKB data (the latest ShareVDE data)
  - Besides report, will work on bringing above work to work relationships, Wikidata queries Tim and Steven had reviewed (for properties not usually found in our catalog) and IMDB connections into demo
  2022-01-28
  - Huda extending data source writeup and will spend a couple of days building a tool to characterize/explore data in DAVE
  - What past research is there on works and what is useful to users?
    - K Coyle and K Godby work from a few years ago suggested ways to find particular expressions (bf: works) useful, works (bf: hubs / opera) help users get there
    - Steven to check past D&A user studies
DAG Calls
- - - - No institutional memory or documentation of tests for "Other forms of this work"
- 2022-0102-2811
  - Huda has works on HTML/Javascript graph explorer code and showed demo. Steven notes how this will be useful to explore datasets. Where to take this work? Perhaps discuss with Dave based on the set of queries used, and to see whether this can run over his Lucene/ES index for more responsive.
  - Huda working to get counts of items under the same LC Hub. Steven and Huda will discuss the data further
  - For past research on works and usefulness to users, Huda will also revisit FRBR/BIBFRAME + user research references previously suggested for KULA paper
DAG Calls
- 2022-02-11
  - Upcoming crossover episodes (DAG/WAG) (announcements forthcoming). Also session with archival focus: 3/1.
  - Lots of interest in Cornell deployment. Huda gave presentation at ARLIS wikidata group (questions about connecting images that can't follow wikimedia license), 2/24 giving presntation to PCC/Sinopia interest group; 3/9 ALA Linked Data interface group; 3/11 ALA Catalog form and function groupCalls next Tuesday for DAG, and the following for WAG focusing on DASH! production and experimental work

Linked-Data Authority Support (WP2)

Qa Sinopia Collaboration
- 2022-0102-21 04
  - Met this week to to talk about new vocab requests. General consensus is that there are unlikely to be many requests because we've already hit the big ones. Known current requests that are a priority include new ISNI rdf download and Homosaurus. There are existing requests that we probably should check in on to see if they are still needed and then determine priority (i.e. RBMS, ORCID, DCMI, CCL, GAMECIP, OLAC's Videogame Genre)
    - Steven will review additional suggestions and discuss with Nancy
  - Dave has already created a triple store of the new ISNI RDF download including a Fuseki end point. Steven did the review for extended context, https://github.com/LD4P/qa_server/issues/14#issuecomment-1018554436. It is in Dave's court for generating the index. Want support for searching by URI.
  - Dave is looking to switch indexing from Lucene to Elastic Search. He wants to do this before starting the containerization process.
    - Lynette to discuss ongoing maintenance issues for ES vs Lucene, we have mostly Solr expertise in CUL
  - Dave believes he fixed a race condition that was causing the 500 errors. I'm not seeing the EOFError in the logs, but I am seeing RDF::Graph#load failure: IOError. So this may be something else going on.
  - and covered 3 topics:
    - pagination stopped working in Sinopia probably due to a bug fix in the cache system. Previously, the number returned would be different than the total number which would trigger pagination to show, but the number returned was greater than the number requested. This bug was fixed. A second bug has the number returned equal to the number requested which makes it look like you have all the results. Dave has a planned fix.
    - Standards committee may be making vocab requests, but don't expect many. Definitely requesting homosaurus.
    - Renamed existing ISNI auth to ISNI_LD4L_WRAPPED. Used the existing name, ISNI_LD4L_CACHE for the new cached RDF download. These updates are in production.
- 2022-02-11
  - No meeting this week.
  - Jim Hahn asked "if an NAR (name authority record, ref. to LCNAF) is updated, when do we expect to see that change populated through QA?" My response was that we currently take a full dump periodically, but it is not on a predictable schedule. Not the best answer. As auths adopt the working group recommendations, it sets the stage for Dave to use incremental updates on a regular basis (e.g. nightly).
  - Current priorities include
    - fix of total_number_found to make pagination work to get pagination working again in Sinopia
    - add homosaurus authority
    - Update pagination in QA linked data module to use json-api as output
    I reported suspicious access to SYSOPTS (e.g. requests to wp, python, .env, etc.) More info in at DLITSYS-4557. No response from SYSOPTS yet.
Best Practices for Authoritative Data working group (focus on Change Management)
- 2022-0102-2104
  - We meet next Monday.
  - There was discussion in Slack about the usage of Add vs. Create. I believe we have settled on Create indicating the entity is brand new and Add indicating that the entity wasn't available and now it is (e.g. permission change, temporary removal reinstated, etc.) I expect Remove and Delete to follow a similar pattern.
Containerization
- 2022-0102-11
  - Working on external documentation for EMM Change Document API. This was running just on my machine, but it is now available at ld4.github.io, which will make it easier for the working group to review. I am incorporating feedback from the last working group meeting.
  - Coming soon: Simple working examples at ld4.github.io. Dave plans to work on connecting to LCNAF's feed as a proof of concept.
Containerization
- 2022-02-11
  - Chile and Antarctica, yay!
  - Delivered CloudFormation templates to Stanford, they had only partial success, got stuck and then moved to other work. Next work cycle much later in the year
  - Containerized version of lookup quite stable but looking on changes for customization etc.. Lynette feels good about build and deploy workflow. Need to finish off customization before moving to production
    - Lynette will look at how it would be to move the current version live, before the customization is complete
  - Would like to get started on containerizing Dave's cache – need to discuss with Dave. Plan to set up call with Dave to discuss how to containerize the index
  21
  - Greg out for a month.
  - I checked in with Justin about progress. He stated... "I never got it to work correctly. I think the problem came down to networking."
  - Next up is containerization of cache. Probably starting in Feb.

Page tree

Versions Compared

Old Version 1

New Version Current

Key

Linked-Data Authority Support (WP2)

Other Topics

Upcoming meetings/presentations

Next Meeting(s), anyone out?: