...
Huda Khan, Astrid Usong
July 2022
TABLE OF CONTENTS
- GOAL
- METHODOLOGY
- PARTICIPANTS
- SCHEDULE
- KEY OUTCOMES
- RESULTS SUMMARY
- Prototype (system and/or mockups) results summary
- Production results summary
- TASKS AND SCREENSHOT REFERENCES
- TASKS
- Screenshots from the production catalog
- Screenshots from the prototype
- Screenshots used by the last participant
- SUMMARY BY PARTICIPANT
- NOTES (Huda)
Anchor GOAL GOAL
| GOAL | |
| GOAL |
We wanted to understand if participants could easily navigate to and find the additional relationships we had added to the interface. We also wanted to understand which of the relationships the participants might find useful.
Anchor METHODOLOGY METHODOLOGY
| METHODOLOGY | |
| METHODOLOGY |
We sent out a recruitment email to contacts who forwarded the message to Mann and Olin student employee email lists and to a list for student supervisors. When participants responded, we scheduled a half hour Zoom session with them and sent them the consent form as an attachment. During the Zoom session, after obtaining verbal consent from the participants, we provided a link to the tasks and links to the prototype and production catalog. In one case, due to technical difficulties, we provided a link to slides with screenshots from the prototype. We began the session with a preliminary question about how participants conducted their research and ended the session with a few follow up questions.
Anchor PARTICIPANTS PARTICIPANTS
| PARTICIPANTS | |
| PARTICIPANTS |
Name | Student or staff | Area of study or work | |
01 | Undergrad, rising senior | Human ecology | |
02 | Staff member | Veterinary school | |
03 | Staff member | Collections assistant | |
04 | Undergrad, rising senior | Development sociology | |
05 | Grad student | Public health | |
06 | Grad student | Plant pathology | |
07 | Grad student |
Anchor SCHEDULE SCHEDULE
| SCHEDULE | |
| SCHEDULE |
July 5 - 10
Anchor KEY OUTCOMES KEY OUTCOMES
| KEY OUTCOMES | |
| KEY OUTCOMES |
- The prototype design enabled many participants to find translations, adaptations, and sequels when they either could not complete at least one of those tasks using the production system. . With the production design, none of the participants found the last sequel, and most had trouble finding a translation.
- Participants were able to use the prototype to find availability for particular resources, so the changes in design did not appear to make availability information harder to locate.
- Although most participants were able to eventually find and use the “explore related items” link, the display and labeling for the “explore related items” link could be updated to highlight the function and resulting content. Suggestions include changing the link to a button and changing the language to more clearly reflect that the resulting work page has directly related versions and works.
- Although some participants noted that they do not usually look for sequels, movie adaptations, and/or translations, they found the prototype pages to be easy to use. Examples of other information that may be considered useful were multiple editions of textbooks.
...
Although we did not include tasks focusing on the reordering of information on the item page to display work information first and then instance information, none of the participants discussed the re-ordering or seemed hindered by it when working on their tasks.
Anchor RESULTS SUMMARY RESULTS SUMMARY
| RESULTS SUMMARY | |
| RESULTS SUMMARY |
Prototype (system and/or mockups) results summary
...
- Biggest areas of struggle: finding sequels and translations.
- All seven could not find the last sequel
- Five could not find the translation
- Three could not find the first sequel
- Two did not identify the movie adaptation
- Exhibited combination of search strategies
- Keyword, author facet, language facet, format facet
- Many used the “Other forms of this work” link thinking it could also help them with finding translations and adaptations. Search strategies included using that link and then the following facets:
- Translation search using language facet
- Adaptation search using format facet
TASKS AND SCREENSHOT REFERENCES
- After having selected other forms, some participants exhibited confusion when they tried to add another search term or facet but the OCLC Work ID facet was still selected.
Anchor TASKS AND SCREENSHOT REFERENCES TASKS AND SCREENSHOT REFERENCES
| TASKS AND SCREENSHOT REFERENCES | |
| TASKS AND SCREENSHOT REFERENCES |
We alternated the order of the interfaces the participants tested, with three participants starting with the We alternated the order of the interfaces the participants tested, with three participants starting with the prototype first and three with the catalog first. Due to technical difficulties, we had to provide the last participant screenshots of a previous version of the prototype in lieu of the functioning prototype and they then tested the production catalog like all the other participants.
...
The link to the slide deck used by the last participant is here: https://docs.google.com/presentation/d/1rk-Y0ISIBed56zEbQf0wCeVDxtBbGc3FRHKYf4rkCNg/edit#slide=id.g1368f02aa3f_0_9 and screenshots are included directly below
SUMMARY BY PARTICIPANT
In the table below, we summarize participant behavior indicating which tasks they might have struggled with or not completed using the prototype system or production. For each participant, we also note if they provided specific comments on the prototype or production systems, which tasks they indicated they normally would or would not do with the system, and any features they thought might be useful.
...
No. | Tasks -Prototype | Comments- Prototype | Tasks -Production | Comments- Production | Tasks normally done/not done | Useful features and other comments | |
|---|---|---|---|---|---|---|---|
1 | Did not use the explore link initially when doing Thai translation task. Saw the link when looking for adaptations and then saw translation there as well. | Good to have everything on one page. Suggest adding facets to work page. | Did not find Thai translation or last sequel. | Used “other forms of this work” link to look for translation and adaptation. | Do not do any of the tasks normally but that for looking for availability information. | Seeing if a particular item has online version. Finding translations. | |
2 | All completed without prompts | Did not find first sequel, last sequel, Thai translation, or movie. | Used “other forms of this work” link to try to look for Thai translation, first sequel, and movie. | Do not usually help others to find movies or translations. | “Other copies” is a clear term. Prototype seems clear and intuitive. | ||
3 | Began by searching for and then using WorldCat to look for other copies. Had to be prompted to | “Adaptation” does not reflect regular catalog language. Would help to use language like “format”. Liked the explore/work page. “Explore” and “Browse” seem to mean the same thing. If keeping both explore and “browse related items by call number”, use same language. | Did not find first sequel or last sequel. Did see Dune 2022 in results but was looking for specific version. | Relied on keyword search with facets such as publication year, language, and format. | Do all the tasks except finding sequels. Others that they help are usually grad students or researchers. These people will often look for a particular published year or rare items in the library. | Like the explore/work page which has images. Inclusion of sequels is useful. | |
4 | All completed without prompts | Took a little while to find the translation as defaulted to looking at the left column. Thought “explore related items” sounded like the page would show similar items. | Did not find Thai translation or the last. | Used “other forms of this work” to to find translation using the language facet. Used notes on item page (which gave the name of the first sequel) and keyword search to find the first sequel. | Have looked for other copies and online resources. Don’t usually look for translations, sequels, or movie adaptations. | Translations might be useful. Finding online options is useful. Labeling was easy to understand. Automatically looked at the left column when looking for translations on work page. | |
5 | Successfully clicked on and navigated to the work/explore page but missed translations. Had Tried to search directly and then filter by language facet but had to be prompted to go back to explore/work page to find translation. | Indicated they could perhaps use search or sort by to find first sequel (?) | “Explore all related items” seems to make sense. Perhaps word “all” is not necessary. Didn’t require the arrows next to headings on the explore/work page but those may be useful to others. Missed translations on explore/work page because first only looked at left side. | Did not get Did not open up first sequel record or last sequel, movie adaptation, or thai translation. | Tried to use “other forms of this work” to find Thai translation, sequels, and movie adaptation. | Have looked for other copies or options, online resources, and movies. Have not looked for translations and sequels. | All features were helpful and useful and . Item page with additions is not messy or crowded. |
6 | All completed without prompts | “Super clear!” Perhaps change “explore related items” to “access all”. “Related” does not necessarily seem to imply sequels and different versions. | Initially, did not find first sequel but later saw the name in the notes on the item page. Did not find the last sequel. | Used search strategies such as keyword search and facets | Would not usually look for translations or movies. Would look for different versions of a textbook. | Good to know about translation. Work/explore page is clear. | |
7 | Found the link for work page (labeled “view related items” in the older prototype screenshot) but missed the translation. Had to be prompted to go back to the work/explore page screenshot to find the translation. | Thought that “view related items” would show any items with title word “Dune” in it, similar to sequels. Suggested labeling “view other adaptations and translations” or “View more related information and titles related to Dune”. Suggested making the link boxed to stand out. Noted that listing other copies under the libraries under availability would also be useful. Noted that adding book covers on the item page’s listing of other copies would help differentiate between copies. | Did not find the last sequel or the Thai translation. | Tried to use “other forms of this work” to find translation. | Would not normally look for translations, movies, or sequels | Would consider perhaps not sequels but other editions useful. Would consider sequels useful if reading for pleasure. Wouldn’t consider movies useful |
NOTES (Huda)
Session One
Undergrad, going into senior year, human ecology
...
Go to GoogleScholar or PubMed
—----------
Session Two
Staff member at vet school
...
- Which of these tasks do you currently do when researching or looking for resources? Which of these tasks would you not do? Would you please explain why?
- Don’t often help people find movies, more of a science bend
- Not been asked to help find translations
- Would any of these features be useful to you in how you search or do research?
- Interesting. Page before it has things laid out in intuitive
- Having other copies. “Other copies” is clear
- Do you have any suggestions or feedback about any of the features you saw?
—----------
Session Three
Staff member, assistant for collections
...
- Which of these tasks do you currently do when researching or looking for resources? Which of these tasks would you not do? Would you please explain why?
- Do all of the tasks except sequel
- Users mainly grad students and professors. If they wanted to find in collection, they know a little bit of the area. They know what they want.
- Less sequel
- Know what year they are looking for
- Or use database
- Or looking for rare items
- Might be different
- Would any of these features be useful to you in how you search or do research?
- Like the sequel one (if doing that task)
- Like the explore related items - has images
- “Explore” or browse terms similar
- Probably use the same language - consistent with the catalog
- Do you have any suggestions or feedback about any of the features you saw?
—----------
Session Four
Undergrad, rising senior, development sociology
...
- Which of these tasks do you currently do when researching or looking for resources? Which of these tasks would you not do? Would you please explain why?
- Finding other options for checking out a book
- Also find online resource instead of physical copy
- Haven’t checked for sequels or movie adaptations - associate more with fiction
- Haven’t done translation search but could see it is helpful
- Looking for a text
- Collection or series?
- Other books on same topic by same author
- Collection
- Would any of these features be useful to you in how you search or do research?
- Other options for checking out item
- Find online resource
- Do you have any suggestions or feedback about any of the features you saw?
- Like the labeling of the new site
- Easy to find things
- Tend to go towards the left - sometimes the two columns
- Knowing to check the right
- Maybe looks a little busy
—----------
Session Five
Grad student, public health
...
- Which of these tasks do you currently do when researching or looking for resources? Which of these tasks would you not do? Would you please explain why?
- Would type keyword search - would go to advanced search
- Have done:
- Checking for other options
- Online resource
- Adaptations: other formats
- Movies or something else
- Have not done
- Translations
- sequels
- Would any of these features be useful to you in how you search or do research?
- Yes, all of them
- Definitely other copies section
- Already gives me a view of other copies
- And sequels
- Useful and not messy or crowded
- Q: Is explore all related items ok?
- A: Straightforward
- Clicking on it will take to all related items
- Translations, etc.
- Maybe the word “all” is not necessary
- Self-explanatory
- Work page: anything to add
- The arrows in the nav bar - maybe useful to people who are not tech savvy and may not know that clicking on that would go down
- But may help others
- Do you have any suggestions or feedback about any of the features you saw?
- When first came to this page, only looked at left side
- Arrows unnecessary - b/c people know hyperlink
- Would go past summary - quickly scroll down and ignore the right hand column
- Maybe if nav bar was more noticeable - maybe at the top?
- Putting in buttons at the top
- But if not rushed, a very friendly website
—----------
Session Six
Grad student, plant pathology
...
- Which of these tasks do you currently do when researching or looking for resources? Which of these tasks would you not do? Would you please explain why?
- For personal research, would look for different versions of the books
- Looking for a textbook
- Multiple versions
- Would get the most recent version usually because most up to date
- Nice to see which ones are there to see what is available
- Never looked for translations in catalog - haven’t had the need to do so
- Native language is Spanish. Cool to know you can look for translations
- Movie adaptations
- I only looked for a movie once years ago
- Wouldn’t do that day to day
- But also work in library - very useful to see all the info summarized in a single page
- If people come over and say I’m looking for this book
- Would any of these features be useful to you in how you search or do research?
- Yes, the second page was super clear
- My research does not make me look for books (yes papers, not books)
- But can see improvement
- Can see useful for people whose research is based on books
- Every now and then look for books so can see useful
- Do you have any suggestions or feedback about any of the features you saw?
- Q: Explore all related items
- Access all? Not sure how to say it. Will see every single format
- “Related items”: books are closely related but not necessarily sequels, maybe things from same area or author, but different story
- Could be a little clearer
- Different versions - everything
- Anything to add or take away from explore
- That’s fine
- Pretty clear
—----------
Session Seven
Grad student
Preliminary question
...