...
Huda Khan, Astrid Usong, Steven Folsom
December 2022
TABLE OF CONTENTS
TASKS AND SCREENSHOT REFERENCES
Screenshots from the prototype
Anchor GOAL GOAL
GOAL
| GOAL | |
| GOAL |
We wanted to get participant reactions to our inclusion of Wikidata information directly within the page and through the use of work knowledge panels. We also wanted to understand which of the Wikidata properties the participants might find useful.
Anchor METHODOLOGY METHODOLOGY
METHODOLOGY
| METHODOLOGY | |
| METHODOLOGY |
We sent out a recruitment email to Tracey Snyder, a Cornell music librarian, who forwarded the message to staff and graduate students. When participants responded, we scheduled a half hour Zoom session with them and sent them the consent form as an attachment. During the Zoom session, after obtaining verbal consent from the participants, we provided a link to the tasks and links to the prototype. We began the session with a preliminary question about how participants used the library catalog to look for musical resources and ended the session with some follow up questions.
Anchor PARTICIPANTS PARTICIPANTS
PARTICIPANTS
| PARTICIPANTS | |
| PARTICIPANTS |
Number | Student or staff | Area of study or work |
1 | Staff |
2 | PhD Candidate (fourth year) | Music and sound studies |
3 | Staff |
4 | PhD Student (third year) | Music and sound studies |
5 | Doctoral graduate student (first year) | Composition |
Anchor SCHEDULE SCHEDULE
SCHEDULE
| SCHEDULE | |
| SCHEDULE |
December 19-22, 2022
Anchor KEY OUTCOMES KEY OUTCOMES
KEY OUTCOMES
| KEY OUTCOMES | |
| KEY OUTCOMES |
- Work info buttons appeared easy to find, with only one participant requiring a prompt to look at the included works section of the page. Once the button was clicked, all participants were easily able to navigate to and find the information on the resulting author title browse page. Future implementation could incorporate work info buttons next to included works.
- Catalog numbers provide potentially useful distinguishing information and may be best included nearer the top of the page or next to the title.
- Instead of “codes”, we should use a term more commonly used for this information such as “catalog numbers” or “thematic catalog numbers”.
- If the title is available in other languages, that information may best be positioned very near the title itself, in the languages field on the page, or somehow accessible in the “other forms” of this work page.
- When using asterisks in fields to indicate the data in those fields comes from Wikidata, we could reword the statement about the Wikidata source to say “This information comes from Wikidata” instead of “Some of the information on this page comes from Wikidata”. Additional design options should be reviewed to ascertain how to display Wikidata source information and how these options may be perceived on pages where Discogs information and highlighting are also present.
- Additional research should be conducted to evaluate which Wikidata properties may be most useful for the wider library catalog audience. Among the Wikidata properties we included, catalog number, instrumentation, librettist, and tonality may be candidates for most relevant information to include for a musical recording or score. Historical information, such as date and location of first performance, was deemed potentially useful for specific cases of research but not necessarily for the participants themselves.
- Another potential area of exploration may center around the use of the information captured in thematic catalogs, such as chronology of works and snippets incipits of music for a particular work, to support discovery in library catalogs.
Anchor RESULTS SUMMARY RESULTS SUMMARY
RESULTS SUMMARY
| RESULTS SUMMARY | |
| RESULTS SUMMARY |
- All participants were able to complete the tasks and find the information we asked them to identify in the prototype. Three participants completed all the tasks without any additional prompts. One participant had to be prompted to look at the included works section to find the work info buttons. One participant had to be prompted to continue to the end of the page to see the librettist information, which they also identified within the contributors section at the top of the page.
- Work info buttons appeared easy to find, with only one participant requiring a prompt to look at the included works section of the page. Once the button was clicked, all participants were easily able to also go to the full information page.
- Although participants were able to correctly answer all the questions regarding catalog numbers, one participant noted that “codes” was not how they would usually refer to this information. “Catalog number” may be a better label.
- When asked which Wikidata properties they thought might be useful based on what they had seen in the prototype, all participants indicated catalog numbers were helpful to include. Three participants stated instrumentation was useful information. Three said information about the librettist was useful, with one of these participants saying the information would be useful if they were searching by the librettist. Two indicated tonality may be useful information to include, with one specifying how some pieces of music are identified through their tonality. Another participant stated tonality information was of mixed relevance, as the tonality may be information that could be contested or debatable. One participant mentioned Opus numbers as being useful to include. It is important to note that participants were not given the whole list of properties again when answering this question but asked to react to what they had already seen while completing the tasks.
- Two participants identified that the asterisk at the end of certain fields meant that that information was coming from Wikidata. The other three participants first saw and commented on the Discogs information statement. One of these participants suggested different wording for the Wikidata acknowledgment, replacing “some information” with “this information”.
- While answering follow up questions, all participants indicated they had some familiarity with the concept of catalog numbers. Two participants indicated they were familiar with some types of catalog numbers but not all of them.
Anchor TASKS AND SCREENSHOT REFERENCES TASKS AND SCREENSHOT REFERENCES
TASKS AND SCREENSHOT REFERENCES
| TASKS AND SCREENSHOT REFERENCES | |
| TASKS AND SCREENSHOT REFERENCES |
We shared the following document with the participants with the tasks and questions and links to the prototype. We are also including the text of the questions and tasks below. Screenshots from the prototype and production catalog are included below for documentation.
TASKS
Preliminary questions
Tasks
Follow up questions
|
...
SCREENSHOTS FROM THE PROTOTYPE
Figure 1: Top of the page for Task 1. "Sonata in A major, for violin (or viola) and piano".
...
Figure 12: Author title browse page for "Stravinsky, Igor, 1882-1971. | Ebony Concerto" for Task 4. This page shows the inclusion of information from Wikidata for first performance date, first performance location, and instrumentation. Clicking on "View full info" in the knowledge panel shown in Figure 11 leads to this page.
Anchor SUMMARY BY PARTICIPANT SUMMARY BY PARTICIPANT
SUMMARY BY PARTICIPANT
| SUMMARY BY PARTICIPANT | |
| SUMMARY BY PARTICIPANT |
In the tables below, we summarize participant behavior and answers to follow up questions. “#” denotes the column for participant number.
# | Normal search method | Tasks | Comments |
1 | Cornell library home page. Type in composer and title or short form. | Completed all but task 3 without prompts. Task 2 (Die Zauberflote) comments:
Task 3 (Four sonatas for violin and continue): Prompted to look at included works section. Then, found work info button.
| Quote: “You do my research for me. That’s marvelous” Quote: “More information is always better than less” |
2 | Youtube. Don’t normally search catalog for recordings. Would use title when searching. | Completed all tasks without additional prompts Task 1 (Vivaldi sonatas): Found RV but didn’t know what it stood for. Task 4 (Stranvinsky): Found information in table of contents first and then quickly found work info button below. | Easy enough to figure out. Catalog numbers useful when title isn’t particularly useful. K number then becomes part of title or alternate title. |
3 | Search by title, composers, specific edition, performer (especially for recordings) | Completed all tasks without prompt except Task 2. Task 2 (Die Zauberflote): Saw librettist in contributors and other roles. Prompted to scroll down and then saw librettist also listed there. | Quotable/referenceable: Catalog numbers are confusing. Having laid out all the numbers is illuminating to someone trying to get a handle. |
4 | Composer and name of recording, or keyword search | Completed all tasks without additional prompts. | Work info button solved information overload problem in a way that still makes that information accessible. Important to know source May trust transcription of information from CD more than publicly editable database. Translations of titles is useful, but unclear how to implement this feature. Non-catalog resources: if widely held knowledge, would use Google search. If more esoteric, may use online encyclopedias such as Grove music or Oxford music online to find information like first performance data or historical information. |
5 | Cornell library website using keyword, title, composer | Completed all tasks without additional prompts | If could search by ensemble type and years, that would be a big help. |
Table 1: Summarizing answer to preliminary search question, tasks, and additional comments by participant
# | Tasks realistic | Languages | Wikidata properties | Catalog numbers | Useful features | Suggestions |
1 | Conceivably do all the tasks, with most basic ones with greater frequency. Not many patrons ask for first performance information. Query for librettist possible. | Don’t expect language to be in traditional library catalog page. Alternate forms in card catalog for title. Search by uniform title? | Don’t normally look for non-bibliographic info but useful to have. Catalog codes are bibliographic. Librettist. | “Catalog numbers” preferred term. Used for cross-references. Kochel put works in chronological order, so useful info. Also catalogs can include |
incipits of music with each title and bibliographic info like editions, dates of composition, performance, publication, etc. Useful contextual information to get a handle on the entry. | First performance location may be useful for focused research. Thankful information available quickly and reliably. | Online catalog can be considered as a study guide. | ||||
2 | Don’t generally work with classical music. Don’t often use opus numbers/k numbers, etc. Closest task: looking for book editions, first volume, when published, translation. | Would expect to find other language titles as close to the German title as possible. Title line or language field. When assigning to students, would want English title even if original in German. For self, would want original language title. | Clearly identified asterisk to indicate source. Catalog numbers useful. Tonality and instrumentation helpful. If cared about history, then historical info also useful. | Familiar with some nut not all. Know about K. Use catalog numbers to distinguish between pieces and understand chronology and grouping. | Click to get more info allows initial entry to be less info heavy and then info that don’t often need can be visible later. | K number wasn’t listed as K (said “Kochel” instead). |
3 | Don’t need date of first performance. RV and Fanna numbers may be useful for ordering pieces. Librettist may be useful for cataloging. Mostly check for different performers of work since people want specific recording. Contributors list useful. | Next to the title with a little link. Link between recording and other language titles? Other forms of this work may be a helpful place to put these other language titles. | Saw and noted Discogs source first. Unclear if they identified the asterisk as mapping to Wikidata as source. K number very useful. Instrumentation. Catalog numbers useful. First performance date is interesting for researcher but not for me. | Familiar with them. Useful to determine which piece you are looking for. | Use Discogs all the time. Good for modern music and soundscape music. Think different languages for title would be helpful. | Discussed the use of the subject line on the item page e.g. clicking on “Sonatas (violin and piano) -> scores and parts”, could see what’s in the catalog, filter by composer, and look for scores. |
4 | Would look for catalog numbers as do use library catalog for RV/Opus and other catalog numbers. Would not look for performance date or expect information would be there. Do not typically look for librettist in catalog but may use other sources. | Not sure. Instinct to look on Mozart info page to look for metadata field with language info. Possibly look at “other forms of this work”. (Clicked on “other forms” and looked at language facet). | Can see Discogs source. Indefinite wording for “some” of this information. If the wording had been “this information”, would have looked at the asterisk. Glanced off the asterisk since the wording was the same as Discogs. Location of recording may be relevant for very specific research question but not otherwise. Catalog numbers useful for translating between different performance editions. In manuscripts where names can be inconsistent (e.g. diacritics), would depend on catalog numbers for search. Tonality may be interesting as some, though not majority, of pieces identified solely through tonality. Instrumentation info is often inconsistent and would not expect to see this in the library catalog page. Librettist may be useful if searching for pieces by a specific librettist, otherwise would go to other sources to find this information. | A given piece may have multiple catalog numbers. Having a central repository which provides all the catalog numbers for a given piece. Would search for catalog number they are familiar with. Tends to be vernacular agreement (agreed upon shorthand) regarding which catalog number to use. | (Covered in other answers) | Contributors list has long strings of info. “Other contributors” is a partial list followed by “more” link and that feels more user-friendly. Like the work info button approach, although found it confusing at first since not used to it in the library catalog. |
5 | Look more for new music (recent composers) and less older music like Mozart, Don’t use catalog numbers as much as peers although have looked for alternative catalog numbers. Searched for librettist. Probably use first performance of work. | Helpful to see other titles by name or copied down in notes or “other titles” field. Useful if system search understood the mapping between titles in different languages (i.e. searching for “magic flute” would still direct to “Die Zauberflote”). | Correctly identified asterisk as corresponding to Wikidata source. Useful: First performance date, librettist, catalog numbers, first performance location, opus number. Instrumentation is most important piece of information. Tonality is a mixed bag. | Familiar with K and |
Bach numbers. Otherwise use infrequently. | Catalog numbers, first performance date, location, librettist info all useful. Would use codes to cross-search elsewhere. Search by dates quite a bit. Sometimes search by language if performance sung in another language. | Great work! |
Table 2: Summarizing answers to follow up questions by participant. —-------------------
Anchor NOTES NOTES
NOTES (Huda)
| NOTES | |
| NOTES |
Participant One
Grad/Staff:
Preliminary questions
...
- Were you familiar with the idea of catalog codes (such as Kochel)? If so, how do you typically use these codes?
- Another important use to actually see titles connected to the musical themes. Most catalogs actually print snippet incipit of music with each title. 41 symphonies of Mozart all called “symphony”, handy to have that opening couple of bars of music. Yes, that’s what I’m looking for.
- More bibliographic information in nature. Each entry for each work will give editions, dates of composition, dates of first performance, publication. Information one needs to get a handle on thing.
- Didn’t realize word “codes” was the keyword. Certain catalog numbers. Cross-references from one catalog to another.
- Most major composers/scholars have made scholarly catalogs
- In the library, couple of rows of thematic catalogs. Easily 500 - 800.
- In some cases, composers have multiple catalogs (e.g. Vivaldi). Even in the case of Mozart, standard Kochel, but made his own, but it’s incomplete and not scholarly.
- Catalog number: Mozart’s. Kochel tried to put everything in chronological order. And also mistakes made. Important facet.
- Would any of these features be useful to you in how you search or do research?
- One wouldn’t expect to find it in library catalog. Once it’s there, thankful for it. Now you know where you can find it quickly and reliably. Don’t have to worry about looking up again.
- More information is always better than less
- Certainly possible to imagine a situation - Stravinksy example. Maybe someone looking for Stravinsky premiered in NY. Saw that piece was performed in Carnegie Hall. Could be important to someone doing a highly focused bit of research.
- Do you have any suggestions or feedback about any of the features you saw?
- One might consider online catalog as a bit of a study guide. Can go to it for ideas of things you might want to know or need to know that you didn’t think of yourself. Started for research projects.
...
Participant Two
Preliminary questions
- If you are a student, what is your year and subject area?
- 4th year Phd Candidate, music and sound studies
- How do you normally search for musical scores or recordings using the catalog?
- Truthfully don’t use scores. Usually don’t search for recordings in catalog. Usually by title when searching
- Youtube generally for searching
...
Those numbers are particularly useful when the title isn’t particularly useful. In those cases, the K number becomes part of the title. Alternate title.
...
Participant Three
Preliminary questions
- If you are a student, what is your year and subject area?
- Staff member
- How do you normally search for musical scores or recordings using the catalog?
- Search by title, composers, specific edition, performer (especially for recordings)
...
- Which of these tasks do you currently do when researching or looking for musical resources? Which of these tasks would you not do? Would you please explain why?
- Contributors list would be useful to make sure have right performer
- Or have more info, have the right edition of something
- Less research but more about ordering and cataloging information
- Don’t think need to know the date of the first performance for my work. Really interesting that you can get there
- RV numbers and Fanna numbers might be useful in ordering, to make sure am getting the correct piece
- Librettist: might use that for cataloging work
- Mostly checking on different performers of this work, because people want to hear specific recording of a particular work
- “Die Zauberflote” on the page http://dev-hjk54.library.cornell.edu/catalog/10646838 may be known by other languages.
- Helpful to have it up at the top, next to the title with a little link. That’s where I would be looking for a title
- If had no idea about music, and didn’t know it was magic flute, would be helpful to see it. Would do keywords and that to do a different search.
- Would it be linked to the recording with the other language titles? If it said Magic flute, would that link to other titles?
- Other forms of this work is helpful
- Where, on this page or elsewhere in the catalog, might you want to see titles in other languages for a musical score or recording? How might this information help with searching or viewing musical recordings in the library?
- Which Wikidata information in the prototype would you find useful while searching for musical recordings or scores in the catalog (for example, catalog codes such as K number or RV number)? Which information would you not find useful? For both questions, can you please explain why.
- All the catalog numbers are confusing, (not the interface). Having laid out is illuminating to someone trying to get a handle
- Saw Discogs first
- Obvious that data coming from Wikidata?
- Majority of that data is not coming from Discogs
- Kochel number is useful - but feel like that should be elsewhere very boldly
- First performance date really interesting - for a researcher. Not useful for me.
- Instrumentation is useful. All of it is useful for determining which one you have, especially with Vivaldi, can get confusing .
- Were you familiar with the idea of catalog codes (such as Kochel)? If so, how do you typically use these codes?
- Familiar with them. Most useful to determine which piece you are looking at or trying to find. Gets confusing certain composers have a number of catalog numbers.
- Would any of these features be useful to you in how you search or do research?
- Accurate when researching modern music and soundscape music.
- Personally use Discogs all the time. Interesting they are supplying info.
- Different languages for title: personally think that would be helpful. Especially if someone
- Do you have any suggestions or feedback about any of the features you saw?
- Subject is useful if they want to look for other
- Could see what’s in the catalog
- Filter by composer
- For scores
- If looking for particular score to play it, if clicked on sonata, would it be listing
- Subject: sonatas, shouldn’t it take me to that whole list on
- Using the actual subject line (Sonatas (violin and piano) -> scores and parts)
- Subject is useful if they want to look for other
- Don’t think so
- Like how links are in blue
- What other things could I do if just looking at sonata record (Vivaldi sonatas in A major for violin and viola)
...
Participant Four
Preliminary questions
- If you are a student, what is your year and subject area?
- Third year PhD student in music and sound studies
- How do you normally search for musical scores or recordings using the catalog?
- Typically by composer and name of recording of name of piece, just as general keyword searches
...
- What kinds of resources do you use other than the catalog and why do you think reliable?
- Grove music or Oxford music online: first performance data, historical information
- Widely held knowledge, may just do Google search
- If more esoteric, online encyclopedias
...
Participant Five
Preliminary questions
- If you are a student, what is your year and subject area?
- First year doctoral graduate student, composition
- How do you normally search for musical scores or recordings using the catalog?
- Usually go to directly to Cornell library website and search in that first search box
- Usually keyword, know the title, maybe the composer
...