Usability
Analysis of
Worldcat
beta
the challenge
The J. Murrey Atkins Library at UNC Charlotte planned to implement WorldCat Discovery in place of WorldCat Local. To prepare for this transition, the project group in my class, ENGL 4181: Writing User Documents, designed usability tests were in hopes of challenging problems from the old interface and reveal additional problems with the Discovery interface. We submitted our findings, and design and language recommendations to the Usability Coordinator and team at Atkins Library.
at a glance
Client: Atkins Library
My role: Usability Researcher
Duration: 3 weeks
Tools:
​
​
Research questions
-
What are the inconsistencies in the beta interface?
-
What are areas of frustration or success in the beta interface?
-
What language and design changes are needed to make the interface usable?
the approach
We collected two types of data:​​
-
Think-Aloud Protocol with students to articulate their actions, thoughts, and observations as they completed real tasks using the beta interface
-
User Interviews in which both the focus group and think-aloud protocol users answer questions about and elaborate further on their observations and opinions
​
We developed the task list after exploring the beta interface ourselves, identifying some of the common tasks that a student might perform. We ran a sample test on a willing room mate, made the necessary changes, and scheduled the tests. We recruited users on a personal basis, in total scheduling five participants. Before the tests, we gathered basic demographic information on the participants. Then, users completed a series of tasks locating and searching with the interface. We asked a number of follow-up questions, and thanked participants for their help with improving the beta version. Afterwards, we met as a group and discussed our findings.
​
The data analysis portion of the project involved the group coming together to summarize participants comments and observations, alongside our own, and co-write recommendation report. We provided mock-up designs of changes to the interface itself.
constraints & solutions
This group work was particularly challenging, because my members were sincerely uninterested in usability testing. Keeping the group motivated was my main priority, and I wanted to ensure that the quality of the tests, data, and recommendations weren't being hindered by disinterest. As a result, I recommended that we, if possible, all be present for the tests and discuss the findings as a group after each test. This ended up working well for us, as my other two group members fell into the roles they liked best (I took turns with one member facilitating, and the other group member preferred to take notes and observe).
​
Working as a group to write a report was a constraint in itself, as we had varying styles and assigned different portions of the report to each member. I tasked myself with trying to ensure that the report flowed without abrupt changes in the writing, and that our tone and style was consistent.
​
A constraint, which wasn't a major concern of the library, was that our participant numbers were not methodologically ideal. Studies from Nielson have shown that 8 members can identify 90% of the usability issues in a system/product, and I prefer to aim for around 8-12 to ensure that the data is statistically sound. However, this wasn't a major concern of the library, because there was one other group in the class conducting usability tests on the same system, but with their own task list, different participants, and results.
the outcome
Our participants were all able to complete tasks in under 10 minutes, and the issues they encountered were pretty consistent. We divided our findings into two sections: Layout and Design, and Content, Language, and the User. We included screenshots marked up with the areas users experienced frustration or struggled to complete the tasks. In our recommendations section, we provided mock-up design changes. We submitted our findings to the usability team at J. Murray Atkins Library.
​
The new WorldCat Discovery is now live on the J. Murray Atkins Library site.