Annals of Family Medicine: finding where a medical journal's website gets in readers' way
Clinicians, researchers, and students rely on Annals of Family Medicine for the latest research. We audited the journal's website to find out where it slows them down, and how to fix it.
Overview
Problem
The journal wanted an impartial review of its website: who its readers are, where the experience creates friction, and how to strengthen its communication with readers.
Approach
We layered six methods, from interviews and a heuristic evaluation to a piloted survey and remote usability tests with five medical professionals and students.
My role
UX researcher on a five-person team. I helped plan and pilot our research, moderated one of the five usability tests, and co-authored the survey and usability testing reports.
Outcome
A prioritized set of recommendations on search, navigation, labeling, and accessibility, backed by severity ratings for every task and participant.
- Timeline
- Winter 2023
- Team
- Five UX researchers
- Client
- Annals of Family Medicine
- Methods
- Interviews, interaction map, comparative analysis, heuristic evaluation, survey design, usability testing
The challenge
Annals of Family Medicine is an independent, open-access, peer-reviewed journal. Its readers are clinicians, researchers, and students with little time to spare, and the journal wanted a more modern website with less friction.
We framed the project around three research questions:
- Who are the website's readers?
- How can their experience on the website be improved?
- How can the journal improve its communication with readers?
How might we help busy clinicians, researchers, and students find, read, and respond to research without fighting the website?
Research approach
We built each method on the one before it, moving from broad understanding to direct observation.
Interviews
Four interviews with clinicians and students gave us a first look at how readers use the site.
Interaction map
Mapping how pages connect revealed unintuitive paths before we tested with anyone.
Comparative analysis
Five competing journal websites showed us industry standards, especially for search.
Heuristic evaluation
Nielsen's 10 usability heuristics surfaced issues like weak system status and error prevention.
Survey design
A survey to reach more readers, piloted and refined before handing it off.
Usability testing
Five remote sessions watching real readers complete six key tasks.
Designing the survey
Interviews gave us depth, but we needed a way to hear from more readers. We designed a survey around our three research questions, covering who readers are, why they use the site, and how they hear about the journal.
We recommended stratified sampling, recruiting equal numbers of researchers, students, and clinicians, because each group uses the site differently and random sampling could over-represent one of them.
What piloting changed
Each of us ran a quick pilot with one person, five in total, asking them to think aloud as they went. The pilots exposed real problems, and we revised the survey before finalizing it:
| What we found | What we changed |
|---|---|
| Two questions asked about occupation in different ways | Merged them into one question |
| Too many open-ended questions, especially about why people use the site | Converted most to single- or multiple-choice questions |
| Answer options didn't cover everyone | Added an "Other" option to several questions |
| Some questions were vague, like not saying which journal "reading time" referred to | Reworded them to be specific but still concise |
Usability testing
Participants
We recruited five participants through contacts the client provided. All worked or studied in medicine, and they ranged from long-time readers to first-time visitors.
- P1
Medical student
Uses the site every few months, usually from email alerts
- P2
Family physician
Background in computer science; searches the site quarterly
- P3
Physician in Brazil
English is a second language; edits similar journal websites
- P4
PhD student
First-time user; has used similar journal sites
- P5
Kinesiology student
First-time user; new to the medical field
Setup
Sessions ran fully remotely over Zoom. In most sessions, one teammate moderated while another took notes in a shared Google Sheet, rating every task's issues on a 1 to 5 severity scale. Participants thought aloud as they worked through six tasks that formed a story: finding an article, searching for more, digging into its data, responding to it, following its citations, and volunteering as a reviewer.
Piloting the tasks
Our pilot test showed the first task was too complicated, so we simplified it and gave it a more realistic reason:
Before
"You just finished your today's work and decided to see if there is some updated research published... Start from the index page and try to find the current issue page. Then find an Original Research paper, read its results and download its pdf."
After
"You have a flight later in the week and would like to read some research during your flight. Find an Original Research article from the current issue of Annals of Family Medicine, and download it as a PDF."
The new version cut unnecessary steps and explained why someone would download a PDF in the first place.
Results
Every participant completed the first task without trouble, but other tasks told a different story.
| Task | P1 | P2 | P3 | P4 | P5 |
|---|---|---|---|---|---|
| Task 1 Download a PDF from the current issue | 1 | 1 | 1 | 1 | 1 |
| Task 2 Search with filters (advanced search) | 4 | 4 | 4 | 3 | 3 |
| Task 3 Find an article's figures and data | 2 | 3 | 1 | 1 | 1 |
| Task 4 Post a response to an article | 3 | 5 | 1 | 1 | 4 |
| Task 5 See who has cited an article | 1 | 1 | 1 | 1 | 2 |
| Task 6 Find information on becoming a reviewer | 1 | 2 | 1 | 4 | 1 |
Overall ratings varied widely. Most participants found the site generally user-friendly, but the participant who struggled most rated it a 1 out of 5.
What we learned
Advanced search was hidden in plain sight
Every participant had trouble with the filtered search task. Even a long-time reader never found the advanced search, and others had trouble setting a date range.
“eLetters” didn't mean “respond”
Three participants struggled to post a response. The tab name didn't suggest responding, and one participant searched the footnotes and sidebar, expecting responses at the end of the article.
Too many “cited” links
One participant found several places on the page mentioning citations, then checked the references section instead.
An overwhelming navigation bar
One participant spent a long time looking for reviewer information because the navigation bar held too much.
Readers want figures in context
Two participants skipped the Figures and Data tab and downloaded the PDF instead, preferring to see figures next to the text that discusses them.
Colors were too similar
One participant liked being able to preview abstracts but found the colors too similar to tell sections apart.
eLetters does not make me think 'response'.
Why would they design the search this way?
Recommendations
| Recommendation | Why |
|---|---|
| Bring search filters forward | Replace the separate advanced search page with filters in a dropdown, suggest keywords, and let readers save searches instead of re-entering everything after one change. |
| Rename and move responses | Place responses at the end of each article, or rename "eLetters" to something like "Responses." |
| Use "cited" once | Limit how often "cited" appears on a page so readers know exactly where to look. |
| Simplify navigation | Group the navigation bar into fewer, clearer categories. |
| Guide reviewers | Give clearer, more direct instructions for becoming a reviewer. |
| Improve readability | Make text inputs and messages readable, improve color contrast, and keep design elements consistent across the site. |
Accessibility
Accessibility in this project
Accessibility came up throughout the project. Our heuristic evaluation found the site harder to use for people with visual disabilities and people less comfortable with technology, and a usability participant found the colors too similar to tell sections apart.
We recommended improving color contrast, offering larger text options, and using clearer labels. We also framed the survey so the journal could learn more about its readers' needs, and give accessibility more weight if many readers reported vision challenges.
Reflection
Piloting paid off twice. It caught an overcomplicated task before our real sessions, and it turned a survey full of vague, open-ended questions into one that would produce usable data.
The project also showed me the limits of a small study. Five participants revealed clear patterns, especially around search, but a larger test would show how widespread these problems are. The natural next step would be to redesign the search and response features, then test again to confirm the changes help.