Usability
Evaluation
of the AIDSinfo
and in foSIDA
Web Sites
Submitted by
Usability
Evaluation
of the AIDSinfo
and infoSIDA
Web Sites
Contents
AIDSinfo Report ........................................................................................................................... 1
Background Information ............................................................................................................. 1
Study Design/Methodology......................................................................................................... 2
AIDSinfo Screening Criteria ................................................................................................... 2
Warmup: Favorite Web Sites ...................................................................................................... 2
Familiarity with AIDSinfo Web Site ........................................................................................... 3
The AIDSinfo Web Site .............................................................................................................. 3
Findings from AIDSinfo.............................................................................................................. 3
Scenario 1: Locating the Guidelines........................................................................................ 4
Scenario 2: Using the Drug Database to Locate Drug Information ........................................ 8
Scenario 2a: Using the Drug Database to Locate Clinical Trial Information ......................... 9
Scenario 3: Using the Clinical Trial Search .......................................................................... 10
Scenario 4: Using Health Topics To Locate Relevant Links/Resources ............................... 11
Scenario 5: Mobile Formats .................................................................................................. 13
Scenario 6: Locating Live Help and Contact Information .................................................... 14
General Comments ................................................................................................................ 15
Final Observations and Recommendations ............................................................................... 16
infoSIDA Report ......................................................................................................................... 18
Study Design/Methodology....................................................................................................... 18
infoSIDA Participant Recruitment ........................................................................................ 18
infoSIDA Screening Criteria ................................................................................................. 19
Office Setup........................................................................................................................... 19
The infoSIDA Home Page ..................................................................................................... 19
Findings from infoSIDA............................................................................................................ 20
Locating the Spanish Pages ................................................................................................... 20
Scenario 1: Finding When To Start Treatment ...................................................................... 21
Scenario 2: Locating Guidelines ........................................................................................... 23
Scenario 3: Using the Drug Database.................................................................................... 26
Scenario 3a: Clinical Trial Information ................................................................................. 28
ii
Scenario 4: Glossary.............................................................................................................. 29
Scenario 5: Live Help and Contact Information .................................................................... 31
Mobile Formats ..................................................................................................................... 32
Temas de Salud (Health Topics)............................................................................................ 33
Hojas (Sheets) Versus Pautas (Guidelines) .......................................................................... 34
General Comments ................................................................................................................ 35
Final Recommendations............................................................................................................ 36
Appendices
Appendix A: Participant Screeners
Appendix B: Moderator Guide
Appendix C: Informed Consent Form
iii
AIDSinfo Report
Background Information
AIDSinfo is a U.S. Government source for the latest HIV/AIDS medical practice guidelines,
HIV treatment and prevention clinical trials, and other HIV-related research information.
What began in 1989 as a call center and a small clinical trials database is now a multi-channel
communication project using the Web, call center technology, in-person trainings, and outreach.
In 2007, AIDSinfo launched infoSIDA as a sister site for Spanish-speaking patients and family
members to locate reliable HIV treatment and research resources. Survey data indicate that
patients and family members seeking HIV-related Spanish-language resources are the primary
participants of infoSIDA. Though the AIDSinfo and infoSIDA projects share the same
overarching goal and largely offer the same information, the mission of the infoSIDA Web site
places greater emphasis on providing consumer health information, both for patients and family
members (who are searching for Spanish-language materials) and for clinicians (who often need
complementary materials for their Spanish-speaking patients).
The National Library of Medicine (NLM) contracted with ICF Macro to conduct usability testing
on both the AIDSinfo Web site and the infoSIDA Web site, to obtain specific information for
tailoring the Web sites to the needs of their respective audiences.
Overall research questions included the following:
• Are the navigation, placement of content, and overall organization of the AIDSinfo and
infoSIDA Web sites intuitive and audience-appropriate?
• Are the overall appearance and design of each site audience-appropriate?
• What changes to the Web sites are needed to more effectively serve the information
needs of the AIDSinfo and infoSIDA audiences?
Key questions relating specifically to AIDSinfo included the following:
• Is AIDSinfo effectively meeting the needs of health care providers?
• Can providers locate treatment-specific information with ease?
Key questions relating specifically to infoSIDA included the following:
• Is infoSIDA effectively meeting the needs of Spanish-speaking HIV-infected patients and
their family members?
• Are patients and family members able to navigate to the information that answers
their questions?
• Is the information at the appropriate readability level and easy to comprehend?
1
Study Design/Methodology
Participants for the AIDSinfo usability testing were recruited through Baltimore Research,
a professional market research firm located outside of Baltimore, MD. Nine health care
professionals were recruited, each to participate in a 90-minute testing session. The testing took
place over a 3-day period, occurring on the following dates: December 20, 2010; December 21,
2010; and January 3, 2011. The usability testing took place at the Baltimore Research facility.
The usability testing was facilitated by a trained ICF Macro usability specialist.
AIDSinfo Screening Criteria
A screening questionnaire was developed. The following are some of the requirements for
candidates to participate in the research study:
• The candidate is at least 18 years old.
• The candidate has experience using the computer/internet.
• The candidate has access to a computer at home or work.
• The candidate has not participated in a focus group or usability test in the past 6 months.
• The candidate is familiar with HIV/AIDS as a physician, nurse, physician assistant, nurse
practitioner, or pharmacist.
• The candidate speaks English.
Warmup: Favorite Web Sites
The usability session began with a question on the participant’s three favorite health Web sites.
The purpose of the question was (1) to see if participants use the internet to access health
information and (2) to gain a better understanding of the various Web sites participants use when
searching for health information, especially for their patients. Below is a list of the Web sites
most frequented by the health care professional group participants:
• Academy of Medical-Surgical Nurses (Nurse Practitioners)
• American Society of Health Systems Pharmacists
• CDC.gov
• ehealthMD
• Google
• IDIC.org
• MDConsult.com
• MDlinx.com
• Medscape.com
• OfficeMD.net
2
• PubMed
• UpToDate Inc.
• WebMD
Familiarity with AIDSinfo Web Site
Only one participant was familiar with the AIDSinfo Web site. This participant mentioned using
the Web site a few times to access the guidelines. For the other eight participants, the usability
session was the first time they had seen or used the AIDSinfo Web site.
The AIDSinfo Web Site
All participants began the usability testing on the AIDSinfo home page. To help readers
understand the participants’ use and navigation of the site, the following sections of the report
will feature screenshots from AIDSinfo Web pages. Below is a clipping of the main navigational
section of the AIDSinfo home page.
Findings from AIDSinfo
During each session, the facilitator presented each participant with a series of sample tasks to
complete on the Web site. For the sample task, the facilitator asked each participant how he or
she would search for the specific information requested in the scenario on the AIDSinfo
Web site. ICF Macro designed the scenarios to gain more insight into the experience of using the
site, including how participants search for the most popular information on the site, and how
satisfied they were with the results of their search. The scenarios are presented in the blue boxes
below. The following sections summarize the findings from this series of sample tasks.
3
Scenario 1: Locating the Guidelines
You have a 22-year-old nonpregnant female patient, newly diagnosed HIV-positive. You need to find
out what HIV regimen to prescribe for her. Your patient also does not know anything about HIV and
needs patient information for herself (e.g., what it means to be infected with HIV). Where in the
AIDSinfo site would you go to find information on what HIV regimen to prescribe for her?
The first scenario involved asking participants to find
information on what HIV regimen to prescribe. While most
participants located the “Adult and Adolescent Guidelines”
page by clicking on the “Treatment/Prevention” link listed
under the “Guidelines” section on the AIDSinfo home page,
two participants did click on the main “Guidelines” header.
Another two participants clicked on the “About HIV
Treatment Guidelines” link to access HIV regimen
information. Only one participant used a different navigation
path to find HIV regimen information. This participant
clicked on the “Search the Drug Database” link under the
“Drugs” section on the AIDSinfo home page. Once on the “Drug Database” page, the participant
clicked on the “Atripla” link on the left side navigation bar. From the Atripla factsheet,
the participant scrolled down to the recommended daily dose to find the HIV regimen.
This participant knew to proceed to the Atripla drug factsheet after reading information she
found on the “Recommended HIV Treatment Regimen” AIDSinfo factsheet.
None of the participants experienced difficulties locating the “Guidelines” page. One participant
said, “I would start with guidelines. I would need to know what the guidelines are for treatment.”
Another participant commented that there should be a separate main header on the AIDSinfo
home page for “What is HIV? All You Need To Know.” This participant expressed that the
guidelines were geared toward health care professionals and that a layperson would not know
this information and would need access to this type of information.
Once participants clicked on the “Guidelines” link, they were directed to the clinical guidelines
prototype. The facilitator explained to participants that the current version of the “Guidelines”
page was being revised and that they would be looking at a prototype of the Web page.
4
Once on the “Guidelines” page, participants clicked on “Antiretroviral Treatment Guidelines”
and then selected the PDF full version of the “Adult and Adolescent Guidelines.” All participants
preferred the PDF version of the guidelines. Only two of the participants selected the HTML link
of the guidelines. One participant said, “What is HTML? I assume PDF is something you
download.” Participants commented that they typically access PDF documents because they are
easier to read and to print, and most documents they access are in a PDF format so they are more
comfortable navigating through a PDF document. Another participant said, “PDF also tells you
how big is the document and lets me know if I want to print. It’s just my preference to use
PDF version.”
Participants commented
that the “Adult and
Adolescent Guidelines”
page needed more detailed
explanation on the type of
information found on the
page. A few participants
seemed confused by the
layout of the page.
Participants did not
understand why the page
included guidelines-related
information under three
categories: full HTML and PDF documents, “Tables Only,” and “Panel Roster and Financial
Disclosure.” Another participant questioned the heading “Panel Roster and Financial
Disclosure.” This participant was unsure of what information was under these headings and
asked whether financial disclosure was necessary to include on this Web page.
5
Once the PDF file of the
guidelines was opened,
participants navigated to the
bookmarkers feature on the left
side of the screen. Participants
scrolled down the list to find the
Table of Contents and clicked to
open it. Some participants just
scrolled down the bookmarker list
looking for information on HIV
regimen. The two sections of the
guidelines that most participants
clicked on were “Initiating
Antiretroviral Therapy in
Treatment-Naïve Patients” and
“What to Start: Initial
Combination Regimen for the Antiretroviral-Naïve Patients.” Once they opened the desired
section, participants seemed unsure how to minimize the bookmarks feature. Several participants
asked if the Table of Contents could be minimized.
Participants reacted to the
highlighted text in the
guidelines. Participants
mentioned that they found
the highlighting distracting
and more of a nuisance.
One participant described
the highlighting as
“irritating.” In response to
a question by the facilitator
regarding the purpose of
the highlighting, only one
participant out of nine
realized that the
highlighted text indicated
new or updated
information.
Where on the AIDSinfo Web site would you go to find information to give to your patient?
One of the tasks in Scenario 2 involved asking participants to find information to give to their
patients. Participants struggled to find the patient education materials, in particular the “HIV and
its Treatment” factsheets. Most participants looked for the factsheets on the AIDSinfo home
page. Only two participants were able to locate the factsheets under the “Related Content”
header on the right side of the “Adult and Adolescent Guidelines” page. Six of the participants
6
found the factsheets under the “Health Topics” section on the AIDSinfo home page. For these
participants, the link to factsheets was not readily viewable on the page. Participants seemed
unsure where to go to find the factsheet. They found the link to patient education materials only
after the facilitator directed them to the “Guidelines” page. Several participants recommended
that links to patient education materials/factsheets be placed prominently on the main AIDSinfo
Web site/home page. One participant commented, “Patient factsheets should be on the initial
main page.” Participants seem to agree that the factsheets were “buried” in the “Guidelines” page
and could be easily missed by someone searching the AIDSinfo Web site.
Another participant said,
“The ‘Patient Education
Materials’ link on the
‘Guidelines’ page should be
bigger. It should not be part of
‘Related Contents.’” This same
participant also suggested that
“Patient Education Materials”
be displayed as a distinct
header in a separate box and
that materials be broken down
by topic, such as treatment
options and general
information about HIV/AIDS
under the “Patient Education Materials” header.
Once participants found the factsheets, most agreed the information provided would be useful for
their patients. They described the factsheets as simple, straightforward, helpful, and important
information for patients to have. One participant commented, “This is the kind of information
I give patients. It would be helpful for them.” Another participant said, “The information is nice
and easy to read.”
Two participants expressed concern that some of the information provided in the factsheet was
too technical and suggested simplifying the language used in the factsheet. One participant
commented, “The information on AIDSinfo Web site has to be at a sixth-grade reading level for
my patients to understand it.”
7
Scenario 2: Using the Drug Database to Locate Drug Information
You have determined that the nonpregnant female patient from Scenario 1 needs to take Atripla,
which is a fixed dose combination of Efavirenz, Emtricitabine, and Tenofovir DF. Where would you find
dosing information so that you know how to write the prescription?
Participants were able to use the drug database to locate
the drug information on Atripla. Most participants located
the drug information by clicking on the “Search the Drug
Database” link under the Drugs section of the home page
to access the drug database. Three participants accessed
the drug database by clicking on the “FDA-Approved
Drugs” link.
1
Once participants were on the “Drug Database” page,
they typed in “Atripla” and found the drug information
they needed. Two of the participants were unsure of how
to find drug information. These participants first navigated
through the “Guidelines” section. However, once the facilitator directed them to the AIDSinfo
main page, they were able to find and use the Drug Database.
Participants were
easily able to find
the information on
drug side effects.
Participants had
strong reactions to
the information
found on the
patient version of
the drug factsheet.
A common
response among
participants was
that the
explanation of side
effects was too technical for a patient to understand. Several participants mentioned that terms
such as “fatal lactic acidosis” were too scientific and should be simplified. One participant
suggested the following revision, “NRTIs in Atripla can cause a fatal buildup of lactic acids
(lactic acidosis) and liver disease.” Another recommendation was “acid-based disturbances may
make it difficult for your body to handle some types of acids.”
1
FDA stands for the Food and Drug Administration.
8
Another concern raised by participants was that the list of side effects such as “hallucination,
memory loss, paranoid thinking, and thoughts of suicide” would scare patients and may result in
a patient not taking the drug for fear of experiencing these side effects. Participants
recommended categorizing the side effects into common side effects and rare or uncommon
side effects. One participant suggested including the percentage of people that experience each
side effect.
Scenario 2a: Using the Drug Database to Locate Clinical Trial Information
Where on the AIDSinfo Web site would you look for clinical trial information for Atripla?
Participants were asked to find clinical trial information on Atripla. Once participants were in the
Drug Database record for Atripla, they were easily able to find the clinical trial information on
the drug. Several participants were able to find the information because they had noticed the
header “Clinical Trials” when scrolling down the health professional drug record.
One participant indicated, “Pharmacists do not give out information on clinical trials. Patients go
through their physicians.” However, this participant was able to find drug information related to
clinical trials by clicking on the “Clinical Trial” tab at the top of the AIDSinfo home page.
9
Scenario 3: Using the Clinical Trial Search
You have an HIV-positive patient who is diagnosed with Hepatitis C and who wants to participate in a
clinical trial. Where on the AIDSinfo Web site would you look for clinical trials related to Hepatitis C?
Participants had no trouble finding the “Clinical
Trials” page. Some participants accessed the page by
clicking on the “Search for Clinical Trials” link under
the “Clinical Trials” section on the AIDSinfo home
page. Two participants used the “Clinical Trials” tab
displayed at the top of the home page. Once
participants were on the “Clinical Trials” search
page, all participants used the basic search function
and were able to find clinical trials on Hepatitis C.
Participants were queried on differences between the
following search methods: Quick Search, Basic
Search, and Advanced Search. Most participants agreed that Quick Search is clear and easy to
understand and that it has grouped HIV/AIDS-related clinical trials into topics. Most participants
agreed that it is clear that Quick Search groups HIV/AIDS-related clinical trials by topic to make
it easy to search for a trial.
Participants were able to
refine the search for
Hepatitis C clinical trials in
Baltimore. However, once
they were provided with a
list of clinical trials,
participants had difficulty
finding the contact person
for the study in Baltimore.
After completing the tasks of
finding clinical trials,
participants rated the
understandability and
process of finding a clinical
trial within the AIDSinfo
Web site on a scale of
1 (difficult to understand) to 10 (very easy to understand).
10
Rating Comments
7 Because it didn’t accept Hepatitis C when I typed it in the search function.
8 Once I found the link, I was able to click on the Web site and find the information.
9 Fairly easy, not difficult.
9 Very easy.
9 Pretty easy.
10 Dead easy, but tons of unfiltered information.
10 It’s not difficult. Fairly easy.
10 Easy.
Scenario 4: Using Health Topics To Locate Relevant Links/Resources
Your clinic is currently in need of additional funding to support a new treatment adherence program
that you would like to implement. Where on the AIDSinfo site can you search for funding
opportunities?
For Scenario 4, a prototype of the new design for the “Health Topics” page was tested. Feedback
on the design and layout of this section received mixed reviews. For the most part, participants
found the layout of the page visually appealing. They liked how the various topics were
categorized into different boxes. However, participants indicated that the page lacked a detailed
explanation of the type of information found on the “Health Topics” page.
Participants were asked to rate the organization of the information on the “Health Topics”
home page on a scale from 1 (poorly organized) to 10 (well organized).
Rating Comments
5
The fact that you have to scroll so far down the page to get information may make
someone miss something.
7
Looks good. It’s not completely perfect. Add some information to the “Health Topics”
home page.
8 Some information needs to be cleared up.
10 Pretty well organized.
Navigating to the “Health Topics” section was a challenge for participants. Participants were not
sure where on the AIDSinfo Web site they would go to find funding information. Three of the
physician participants used the search box on the main AIDSinfo home page to find funding
information. The other six participants struggled to complete Scenario 4. Once a participant
landed on the “HIV/AIDS Health Topics” staging page, they did not know where they would
find funding information. Only one participant used the A–Z Index at the top of the page.
Others scrolled down the page but were unsure where to click to find the needed information.
11
Participants seemed
confused about the various
categories listed on the
“Health Topics” staging
page. A common question
asked by participants was,
“What is the difference
between ‘Clinical
Research’ and ‘For
Researchers’?” One
participant described the
“Health Topics” section as
“smorgasbord” of health
information. It’s not clear what information is there. “I would prefer a dropdown key for each of
these headings.” This same participant commented that he would not think to look for funding
under “Health Topics.”
Another participant said, “I’m not sure how patient-friendly the ‘Health Topics’ section is.
They are going to need general patient information. Information found is too complicated for
a patient.”
Three of the participants suggested changing the name “Health Topics” because the information
provided under “Health Topics” was too extensive for the title.
Overall, participants liked the information found on the AIDSinfo Web site. Some participants
expressed concern that some of their patients would find the information too technical.
One participant commented that she would not refer patients to the AIDSinfo Web site because
it’s too “intimidating.” Participants suggested changes, which included adding more instructional
text at the top of the staging page as well as adding a separate tab to indicate what information is
for a health care professional and what information is for patients or the general public.
12
Scenario 5: Mobile Formats
One of the areas we are interested in knowing more about is how providers use their mobile devices.
Do you use a smartphone (BlackBerry, iPhone), iPod Touch, or other mobile device?
In this scenario, participants were queried on their use of mobile phone applications. Five out of
the nine participants indicated they have a smartphone such as a BlackBerry or an iPhone.
Most participants indicated that they prefer using a laptop or a desktop PC to access health
information. The participants who use a mobile device to access health information use their
mobile phone to look up drug information through Web sites such as Epocrates or WebMD.
The feature that most appealed to them about these sites was quick accessibility to the
information they are searching for. Asked if they would consider using a mobile device to access
health information on the AIDSinfo Web site such as the guidelines or the Drug Database,
participants expressed concerns as to the ease and speed of accessing information. However,
they did express an interest in having a mobile-friendly application giving them access to the
guidelines and factsheets, and having an abbreviated version of the Drug Database for health
care professionals.
Time was a big concern for these participants. Participants stated that they are connected to a
laptop or computer at work all day and, given their patient load, they do not have the time to look
up information through an application on their mobile phone. For participants, being able to
access information easily and quickly is important. They would have to consider these factors
when deciding whether to download a phone application.
Participants agreed that the AIDSinfo Drug Database, the patient education factsheets, and the
guidelines would be a valuable resources to access via a smartphone. However, they indicated
the guidelines would need to be condensed because the document files would be too large to
access via a mobile device. One participant suggested having access to certain parts of the
guidelines, such as the summary, diagnosis, treatment, and antiretroviral regimen information.
Another participant recommended providing access to the treatment and prevention guidelines.
The following is a list of other recommendations participants would like to see included in a
mobile-friendly application for AIDSinfo Web site:
• Guideline sections presented in table content format.
• Tables or template of the guidelines. The tables would be listed separately and with
links to the information or advanced search.
• Health care professional version of the drug factsheets. No references needed via the
mobile phone. This would include a list of headings, and a participant would be allowed
to click on the heading to access the information needed.
• Dosing information for various drugs, including side effects and drug interaction.
• Information should be provided in bullet form.
13
• Physician version of the drug factsheet. The factsheet should be similar to what is
available through the physician’s desk reference that would include information such as
adverse reaction, dosage, and FDA drug requirements.
These professional participants also did not find the glossary useful given their familiarity with
HIV-related terms. Participants mentioned that the glossary of terms would be more appropriate
for their patients.
Several participants commented that the usefulness of mobile applications to patients would
depend on the patient’s comprehension and willingness to read. One physician stated, “I would
recommend a mobile phone application to an HIV patient that is very anal about their care—a
patient that is very involved in their care.” Another physician expressed that HIV/AIDS patients
would be reluctant to download a mobile phone application, “Patients would not use it. HIV-
positive individuals are very private about their condition, and the last thing they would do is to
put an HIV/AIDS application on their phone for fear that their peers or family members would
find out about their condition.”
Based on overall responses, health care professionals would be interested in using a mobile
application to access some of the information available on the AIDSinfo Web site. They would
prefer access to the Drug Database and an abbreviated form of the guidelines. Several
participants did find the glossary feature useful for their patients.
Scenario 6: Locating Live Help and Contact Information
For Scenario 6, participants were asked to find the “Live Help” feature. Participants were able to
locate the “Live Help” feature and they understood that this feature would allow them to “chat”
in real time with an individual. Participants questioned if the person responding via “Live Help”
was a health care professional such as a physician or a nurse. One participant said that it was
most likely a “government employee” and not a health care professional. Participants did not
seem to express interest in using the “Live Help” feature. Most agreed that this feature was good
for a patient or layperson seeking more HIV/AIDS information. Participants also noted that the
hours of “Live Help” were limited and should perhaps be extended to a standard business day.
None of the participants located the toll-free number found at the top right side of the AIDSinfo
home page.
Participants located the toll-free number
by scrolling down the page under
“Frequently Asked Questions.”
All participants remarked that the
toll-free phone number was too far
down on the page and should be placed
in a more prominent place on the
Web page in a larger font size.
14
All participants were able to locate the e-mail address and understood that e-mail was the
alternate help option when the “Live Help” and the toll-free number were not available.
Participants rated the understandability, organization, and usefulness of the information found on
the page on a scale of 1 (difficult to understand) to 10 (very easy to understand).
Rating Comments
5
The “Live Help” needs a picture. Anything, someone talking or chatting. Something that
relates to the chat feature.
5
The print is too small. Bold the FAQs. Telephone number and contact e-mail should be
at top of the page.
5
It’s okay. The phone number should be place elsewhere. There should be instructions
on how to begin the chat. Information on the confidentiality should be more prominent.
5–6 Some things are in the wrong place and some things are missing.
9 Well organized.
9
Clear. It breaks the information up by “Live Help” and hours. It gives you the FAQs.
It’s pretty clear.
9 Pretty good. A quick all-in-one page.
9
Well organized. Larger toll-free number and e-mail. Use “Live Chat” instead of
“Live Help.”
General Comments
Participants were sometimes unclear on the role of AIDSinfo Web site. They wondered if the site
was intended for health care professionals or for patients.
15
Final Observations and Recommendations
Here is a list of recommendations based on findings from the usability testing:
• Add detailed information on the “Adult and Adolescent Guidelines” page explaining
organization of guidelines-related information under three separate categories: full
HTML and PDF version guideline documents, “Tables Only,” and “Panel Roster and
Financial Disclosure.”
• Move “Patient Education Materials” from under “Related Content” to a more prominent
place on the page. Under its current location, it is not visible to participants.
• Categorize “Patient Education Materials” into different subtopics. For example, provide
dropdown boxes allowing a participant to click for information on treatment or general
information on HIV/AIDS.
• Do not highlight text in the guidelines; if this is not possible, then indicate on
the “Guidelines” page that the purpose of highlighting is to indicate new or
revised information.
• Add text in the “Guidelines” document on how to minimize the bookmark feature or add
an “Expand All” or “Collapse All” feature similar to what is on the HTML version of the
“Guidelines” document.
• Add a separate section on the main AIDSinfo home page for patient education materials
and include all HIV/AIDS factsheets under this topic.
• Revise the side effects content found on the patient version of the drug factsheets.
Simplify the scientific terms such as “fatal lactic acidosis” and categorize side effects into
common and uncommon side effects.
• To avoid confusing participants, remove one of the search functions under clinical trials.
Quick Search and Advanced Search seem to be search methods most preferred
by participants.
• Change the “Health Topics” section header to something more descriptive of the section
contents. Include information in the header describing the intended audience.
• Revise the subtopic or categories on the “Health Topics” main page and provide more
detailed descriptions for each category. Consider participant suggestion for “having tabs
on the main page and adding dropdown boxes for the subcategories.”
• Place “More AIDSinfo” section under “Health Topics” or add more descriptive
information on the difference between the two sections.
• Offer patients and the general public a mobile phone application to access the “Glossary
of HIV/AIDS-Related Terms and Factsheets.” Offer physicians and health care
professionals a mobile phone application to access the Drug Database and an abbreviated
versions of the guidelines.
16
• Add more information on how the “Live Help” feature works and information on the
intended audience.
• Place the toll-free number and e-mail address more prominently on the AIDSinfo
“Live Help” page or make the font larger so that it is visible to participants.
17
infoSIDA Report
Study Design/Methodology
infoSIDA Participant Recruitment
The infoSIDA usability study participants were recruited through the following coalition and
organizations: the Metropolitan Latino AIDS Coalition (MLAC), La Clinica del Pueblo, and the
“Health Is Life” program of Casa de Maryland. MLAC is a coalition of organizations and
individuals dedicated to HIV prevention, education, and support targeting the Latino community.
La Clinica del Pueblo provides primary health services to the Latino community. The “Health Is
Life” program of Casa de Maryland works to address public health and primary care needs in the
Latino community, including HIV counseling and testing, and health education.
The recruiter, infoSIDA Task Lead Outreach Coordinator Omar Reyes, gave a presentation on
AIDSinfo/infoSIDA services at an MLAC coalition meeting and invited MLAC members to
participate in the infoSIDA usability project. He explained the purpose and details of the study
and collected contact information from coalition members interested in participating. Mr. Reyes
also contacted key people from the “Health Is Life” program of Casa de Maryland and from the
Clinical Department of La Clinica del Pueblo for recruitment purposes.
After reviewing the screening questionnaires, Mr. Reyes selected nine participants and
three alternates.
Participant Type Number Selected
Patient 3
Physician 1
Caregiver 1
Nurse 1
Health Educator 2
Outreach Worker 1
Participants were contacted through both e-mail and phone to remind them about the date, time,
location of the study, and directions to the location where the study was going to be conducted.
All the information that the recruiter collected from the participants has been kept in confidential
files. Information collected from the participants is maintained in confidential files accessible
only by the recruiter.
18
infoSIDA Screening Criteria
The screening questionnaire included general information on the study and 14 criteria questions.
The following are some of the requirements for candidate participation in the project:
• The candidate is at least 18 years old.
• The candidate speaks and reads Spanish (monolingual or bilingual).
• The candidate has experience using the computer/internet.
• The candidate has access to a computer at home, work, library, or anywhere else.
• The candidate has not participated in a study project in the previous 6 months.
• The candidate is familiar with HIV/AIDS as a physician, nurse, caregiver, patient,
outreach worker, or health educator.
• The candidate is willing to spend 1½ hours in the study.
Office Setup
An experienced, bilingual ICF Macro moderator conducted nine in-person interviews at the
ICF International Z-Tech offices on November 8–11, 2010.
ICF Macro employees set up two computers in the interview room: one laptop for the participant
to use, and a desktop monitor for the notetaker observing the interview. The participant,
interviewer, and notetaker sat in one room in the ICF International office suite, while staff
members from the AIDSinfo project and NLM observed from another conference room.
Participant screen movements were captured using Morae, a usability-testing software package.
Observers were able to both watch screen movements and hear the interviews. A Webcam on the
participant laptop captured an image of the participant’s face for observers to see as well.
Simultaneous translation was available for staff members who did not understand Spanish.
The interviewer asked the questions and probes as found in the Interview Guide
(see Appendix B). After asking all of the questions from the Interview Guide, the interviewer
told the participant that she had completed most of the questions in the interview and that she
needed to check in briefly with the client. After receiving feedback from AIDSinfo staff and
the NLM client, the interviewer went back to the participant and asked a series of
follow-up questions.
The infoSIDA Home Page
To facilitate reader understanding of this report, screen clippings from various sections of the
infoSIDA site are featured throughout the report. A screen clipping of the main navigational
section of the infoSIDA home page is presented to provide readers with an overall
reference point.
19
Findings from infoSIDA
During each session, the interviewer presented the participant with a series of sample tasks to
complete on the Web site. ICF Macro designed the scenarios to gain more insight into the user
experience on the site, including how users search for information. These scenarios are presented
in the blue boxes throughout this report. The interviewer asked the participant how he or she
would complete the scenario on the AIDSinfo Web site or the Spanish mirror site, infoSIDA.
The following sections summarize the findings from this series of sample tasks.
Three participants had visited the Web site before the usability testing session. Two commented
that, in general, the site was very good and that it had a lot of information and options. The third
participant had merely visited the site to view the guidelines and offered no comments about
the site.
Locating the Spanish Pages
All participants commenced the
usability test on the English-language
home page of AIDSinfo. Several
participants began to look for
information on the English site;
others immediately moved to the
infoSIDA site. Regardless, all
20
participants used the “Español” button on the top right of the home page rather than the small
“infoSIDA” logo on the upper left to switch to the infoSIDA.
Scenario 1: Finding When To Start Treatment
You are looking for information in Spanish about starting HIV medications. You have come to the
infoSIDA Web site to find this information.
The most direct way to find Spanish consumer factsheets
to respond to the scenario would be to click on “Hojas
Informativas” (Patient Fact Sheets) under the
“Tratamiento del VIH/SIDA” (Treatment) section of the
infoSIDA home page. None of the participants did this.
Instead, five participants (just over half) clicked on the
“Medicamentos” (Drugs) section when looking for
information about starting HIV medications. One
participant explained, “I would go there because I want to
find out about drugs.” However, clicking on
“Medicamentos” takes users to the main landing page for
the Drug Database, not to general information about HIV medications as some of them appeared
to expect. Once participants landed on the “Drug Database” page, many of them looked confused
and did not know where to go, as a drug name is required to find more information from
the database.
For users who began their
search in the “Treatment”
section and users who went to
the “Treatment” section after
looking under
“Medicamentos,” six of them
(over half) clicked on the all-
capitalized red section header
“Treatment of HIV/AIDS.”
Clicking on the header of the
“Treatment” section takes
users straight to the
“Guidelines” page (see
screenshot). From the
“Guidelines” page itself, there
are no links to the Spanish
consumer factsheets, as these
are located on the secondary
pages. One person clicked on the “Treatment” tab on top right of the infoSIDA home page,
which also leads users straight to the guidelines.
21
At this point in the discussion, the interviewer frequently assisted in guiding participants to the
“Hojas Informativas” (Consumer Factsheets) page through the “Hojas Informativas”
subheader/link in the “Treatment” section of the infoSIDA home page. Once on the “Hojas”
page, it was not immediately clear to all users that the individual factsheets were listed on the left
navigation bar. For example, instead of clicking immediately on a factsheet, one patient stared at
the page and hovered the mouse over the left side of the navigation bar until he said, “I get the
idea that these are the sheets.” Another user commented, “What I didn’t pay attention to was on
the left side—the colors aren’t eye catching. I don’t know if this information is important, but I
didn’t notice it—I wouldn’t have found this information.”
Once people got to the information sheets on the “Hojas” page, they appreciated the information
and the way the factsheets were written. They described the factsheets as simple, not
complicated, and written in language that is accessible for the majority of people. One participant
commented, “I like the topics that they had there. They were all very important. I’d like to go
back there.” Another participant described the information as, “Very explicit, very good.
I’m reading it as though I were a patient. Everything is fine. Easy to understand.”
Another commented, “This is excellent. This is talking exactly to me. Those recently diagnosed
with HIV. This is exactly what I was looking for.”
However, a physician who located the factsheet series “HIV and Its Treatment: What You
Should Know” by navigating from the “Adult and Adolescent Guidelines” page commented on
the length of the series:
This is 22 pages. Sometimes that’s too much to read. If this could
be made into a summary of two pages…the most important and
basic questions that would be easier…patients have just received
their diagnosis and are sometimes confused. If we could have two
pages and then say, “If you would like more information…”
Two people suggested that the information be presented more graphically rather than primarily
through text. When presented with a printed copy of a factsheet, one of the patients stated,
“It would get my attention but I would suggest a little visual help. Vignettes or something that
would make the questions pop out.”
22
Another commented, “‘Consulta con el medico del VIH,’ [Seeing an HIV Doctor] I don’t know
what that means. Is it online? Is it going to tell me to go to the doctor? It got my attention
because it says ‘Doctor.’ That’s very important to me.”
The “English” tag following the title of each factsheet (pictured) confused some of the
participants. Several did not understand the purpose of the tag: to indicate that the English
version of the factsheet could be downloaded by clicking on the “English” tag. Instead, these
participants thought the tag indicated that the factsheet was available only in English.
In general, navigating to the factsheets appeared to be the most difficult problem participants had
in finding the information required to address the scenario. This may be in part because of the
generic term (“Informative Sheets”) used as the link on the home page. When the moderator
asked a participant why he/she had not clicked on the direct link to the information sheets,
the response was as follows:
Well, it’s a sheet. For whatever reason it didn’t catch my attention.
I’m looking for treatment. I went to whatever said “Treatment.”
It should have a fuller title, “Hojas Informativas Acerca del
Tratamiento del…” [Informational Sheets About Treatment of…]
It didn’t connect for me. Or have “Guidelines and Information
Sheets About…”
The placement of the Spanish materials at the bottom of the various guidelines pages may have
also made it difficult for users to find the factsheets quickly and easily. Although it is possible to
arrive at some factsheets by clicking on “Treatment” and then “Guidelines for Adults and
Adolescents,” because the Spanish factsheets are featured at the bottom of the page, users with
smaller screens must scroll down the page to find them. This is particularly true if the user is
accessing the site on a laptop or computer with a small monitor. One participant suggested that
the links to the Spanish materials be made more obvious. “When you go to the page, I would
want it to say, ‘Hojas en Español.’ I could go directly to them.”
Scenario 2: Locating Guidelines
You want to know the standard of care for when doctors should begin treating HIV-positive patients.
In response to this scenario, most participants began their search in the “Treatment” section of
the infoSIDA home page. As discussed previously, users entered this section of the Web page by
23
clicking on the “Treatment” headline rather than selecting a subheader. Clicking on the
“Treatment” headline takes users to the “Guidelines” main page. This page features small icons
representing covers of three guidelines and two Morbidity and Mortality Weekly Reports
(MMWRs), and the following instructions across the top of the page: “Select one of the
guidelines (‘pautas’) from the menu to the left.”
Use of the Term “Pautas”
In general, participants did not like the term “pautas” as a translation for “guidelines.”
This became evident at several different points in the interview, including when participants first
clicked on the “Treatment” headline/header. Upon reaching the main “Guidelines” landing page,
three users asked what “pautas” meant. One woman commented that to her a “pauta” was
“un espacio de tiempo. Es como un tiempo de espera.” (“A period of time. It’s like a waiting
period.”) Another said that he had interpreted “pauta” as “más como pausa” (more of a pause).
When the moderator clarified that the term was used as the translation for the word “guidelines,”
the participant looked surprised and asked, “Is that the correct term?”
A male participant commented about the word “pautas,” saying that it “sounds a little odd.”
A female participant who disliked the word strongly said, “I’ll replace it in my head with
‘normas’ [norms] when I read it.” Another participant stated, “I understand that I am going to
have a ‘pauta’ in a conversation, but in this context…I would rather see…maybe something
more specific and direct, less technical.” Another participant said, “I think in this context it’s not
very descriptive. It would be more explicit to say ‘guías’ instead of ‘pautas.’” Still another
participant said, “I still have problems with the word ‘pautas.’ I don’t like it. I don’t know what
the audience is here that you’re looking for. I’d like a different word. I think I understand what
it’s referring to. I’d like something simpler, direct, easy to understand. Something that anyone
would understand.”
The following are alternate terms suggested by participants.
Suggested Alternate Term Times Mentioned
Guías/Guía de tratamiento/Guías para tratamiento
6
Normas
3
Reglas/Reglamientos
2
Información
1
Pasos
1
Use of Icons on “Guidelines” Page
Participants did not like the icons representing covers of the guidelines and MMWRs currently
featured on the “Guidelines” page (see screenshot on previous page). They were frustrated that
the text was too small to read. Asked what he saw on the main “Guidelines” landing page,
one patient responded, “That the letters are very tiny and really hard to read.” One male
participant commented, “You could have an apple there, and it would give me the same
information [as the “Guidelines” icons].” Another said, “I still want to be able to see the titles of
the documents. I still don’t know if this would have the information I’m looking for.”
24
Participants indicated that it would be more useful to have readable text regarding the type of
information contained in the document than a picture of the document.
Clicking Through to the Guidelines
All participants who clicked on a “Guidelines” icon clicked on the icon for the “Guidelines for
Adults and Adolescents.” Upon arriving at the “Adult and Adolescents Guidelines” page, the
participant who suggested that a graphic of an apple would be equally informative stated,
“You have a lot of stuff on here, but it doesn’t say anything. The only thing you can see is on the
page. These other images don’t help. Make it simpler.”
One participant who clicked through to the guidelines themselves commented that he clicked
on the orange and yellow guidelines document because it was in the center of the page and
was colorful.
At least two participants commented on the lack of specificity in the titles listed below each
“Guidelines” icon. A physician remarked, “I would assume that when you refer to adults you’re
talking about adults with HIV. I’m assuming that this list of immunizations is what you should
get and what you shouldn’t [get].” While looking at the page titled, “Treatment for Adults and
Adolescents,” the participant commented, “It’s information I guess on treatment for HIV?
Not sure.”
Language Availability
All participants agreed that it is clear the guidelines are available only in English. However, a
few assumed that they could receive the same information in Spanish if they registered and/or
asked for it. Another participant was under the impression that she had to register to receive
information in Spanish:
You should have all in both languages without asking for it.
If you’re diagnosed, you’re going through something very difficult.
You look for information, but you go to a professional, which in
this case is me. I think this same site should provide the
information to me in Spanish—in the same spot without having to
ask for it.
A minority of participants indicated that having the guidelines only in English was an
inconvenience to them. One participant suggested improving the page by “maybe hav[ing] the
guidelines in Spanish. It makes it complicated that I have to register.” Another commented,
“…as a visitor to the Web site, I’m bothered that it’s not available in Spanish. It’s complicating it
for me.”
Participants were asked to rate the “Guidelines” page on a scale of 1 (difficult to understand) to
10 (easy to understand). The table below presents participant ratings from lowest (6) to highest
(10), the number of participants according the rating, and any comments given as explanation for
the rating.
25
Rating Times Mentioned Comments
6 1
Because there wasn’t a title to look for the information.
I would have to investigate more.
7 2
It’s easy, but not so easy. It’s a little easy because I know
the subject, but not for someone that isn’t familiar.
8 1
[Landing page] is easy to understand. Written simply.
Again, I have a problem with the word “pautas.”
9 1 Because I didn’t understand at first what a “pauta” was.
10 1
Because it’s what you use across the country for
treating patients.
Scenario 3: Using the Drug Database
You are interested in side effect information for Atripla. What would you do to find this information?
The majority of participants used the links in the “Medicamentos” section to find information on
side effects for a particular drug. However, one patient stated that he felt the best way to find that
information would be to look under the “Treatment” section, where he pointed out the factsheets
on side effects. When the moderator asked where he would look for side effects for a particular
drug, he immediately responded that he would look under the “Drugs” section.
Some participants did not understand the difference
among the headers listed under “Drugs” (“Search the
Drug Database”; “FDA-Approved Drugs”;
and “Investigational Drugs”). One participant
commented, “Maybe if I were informed in the subject…
I don’t know how to tell if it’s approved. It’s what could
bring a little bit of doubt.” Another commented, “It’s a
little bit complicated, the difference between these two.
I’ll probably have to look at both. I’ll start with
‘Approved by FDA.’” As he was reading the headers in
the “Drugs” section, one of the physician participants
pointed out that users have to assume that the drugs
listed in the Drug Database are drugs for HIV. Although
the sentence introducing the headers does include
reference to HIV/AIDS, the headers do not include the term.
Five of the participants searched for the drug by clicking on the first header/bullet (“Search the
Drug Database”) in the “Drugs” section. Three participants searched for Atripla using the second
header/bullet (“FDA-Approved Drugs”). One participant clicked on the “Drugs” tab at the top of
the page, which took him to the Drug Database.
The patient who clicked on the “FDA-Approved Drugs” header/link commented that he liked the
listing of all the drugs that came up. He was able to find Atripla quickly. Participants who
clicked on the main “Drug Database” page saw three dropdown options for searching the
26
database: drug name, drug class, and drug approval. Most participants ignored these dropdown
options and simply clicked on the Atripla link in the left navigation bar. The “Drug Class” and
“Drug Approval” dropdowns were confusing to participants who did not click on the left
navigation bar because they knew nothing about Atripla other than the drug name. “What do they
mean, ‘aprobación’? ‘Clase’? I don’t know. Maybe this is the definition…but I don’t know what
classification Atripla falls under.”
One participant who was attempting to fill in the dropdowns accidentally clicked on the “Ayuda”
(Help) link to the immediate right of the medication search box. “I think they should have
‘Search’” immediately after the search box. Put ‘Help’ below ‘Medication’ in case you don’t
know what to put in the ‘Search’ box.”
Once they found and read the information for Atripla, participants rated the understandability of
the information on a scale of 1 (difficult to understand) to 10 (very easy to understand).
Rating Comments
5 Very technica
l.
8 --
9 The first paragraph is conf
using. But the rest I understood.
9 It’s easy to get to the info. It’s easy to know where to click (in search results).
10
Because it talks in gene
ral about the side effects and it gives you information about
clinical trials.
10 Easy to understand.
10
It’s very clear from the beginning. It was a little confusing to get here (9) but once I got
here…it’s very clear.
Users also commented that the information sheet was easy to find, very simple, and written in
big letters and plain language.
27
Where on the infoSIDA site would you look for information on clinical trials for Atripla?
Both physicians commented that they liked the picture of the Atripla pill. “The three names and
the picture of the pill help first off to know that this is three drugs combined.” In contrast,
a patient commented that listing the three medications that make up Atripla is confusing and
unhelpful because the drug names are not clickable. One of the physicians also recommended
that the names of the three medications be clickable so side effects for each of the drugs could
readily be accessed. The same physician commented, “I don’t like that you have to look each
up separately.”
Another physician commented that the information is very confusing. “It’s very technical.
Patients will not understand it. The less you can use technical terms, the better. Some technical
terms you can’t substitute, but some you can certainly simplify.” He suggested having things in
bullets, or an organigram: What is Atripla? Who makes it? What are the side effects? He also
suggested possibly arranging them in tabs at the top of the page.
One of the patients commented that he would like to see side effects featured more prominently.
“Side effects are so important. I would have liked to have seen side effects where it said
‘Medications’ or a more obvious [place]. I don’t want to have to go all around to find this
information. I found it by eliminating things, but this is a very important subject. I would have
liked to have seen this information in a more accessible place or more up front, not making me
go around and around to find it.”
Scenario 3a: Clinical Trial Information
Once participants were in the database
record for Atripla, they were able to
easily find the related clinical trial
information on the drug. Many of the
participants had either already scrolled
past the “Clinical Trials” header or
continued scrolling to find it. Because
the question was asked in the context
of the drug record, it is not possible to
say whether the information would be
equally easy to find when starting from
the home page.
One participant commented that it would be better if scrolling were not necessary, saying,
“I wish it were all up at the top to make it easier to navigate.” A patient commented that although
the clinical trials information was easy to find, it was not necessarily easy to interpret. “[It’s] still
technical language. There are some terms and illnesses that I’m not going to understand. I have
to go to another Web site to find what they mean. It’s a matter of the education that a
person has.”
28
Scenario 4: Glossary
As you were reading the drug record on Atripla, you read that this pill contains three active drugs. Two
of the active drugs are from the nucleoside reverse transcriptase inhibitor (NRTI) drug class and you
want to know more about what this word means. Where would you find more information about the
word’s meaning?
Only one participant found the glossary without assistance. Some participants did not see the
“Glossary” header on the home page under “More Information” because the screen was cut off
before that header could be displayed. Several (four) participants attempted to look for the
glossary in the “Drug Database” section, under the class of drugs tab. One participant tried to
find the answer by clicking on the “Help” button next to the “Class of Drugs.” However,
the resulting page is only in English.
From the Atripla entry in the Drug Database, one of the health educators clicked on the link for
“Inhibidor de la transcriptasa inversa análogo de los nucleosides” in the left navigation bar,
and then commented disappointedly, “But what came out is the names of the meds that have it,
not a definition.” Even when prompted to go back to the home page to see what she could find,
the health education continued to stare at the medications page. “It has to be here in
medications…because I’m looking for information related to medication. But I was already there
and I didn’t find it.”
One of the participants commented, “It’s easy if you look
at the whole page, but you don’t normally think to look at
‘More Information.’” Participants who did look in
“More Information” did not necessarily find what they
were looking for. One male participant read each entry
but still completely missed the “Glossary” heading.
When the moderator pointed the link out to him, he
commented, “This is what I was looking for. ‘Glosario’
should be up here at the top [with the other tabs].”
After having the location of the glossary pointed out to
him, one patient commented, “This is a very good
resource, but look where it is here at the bottom. I have to
go through all this information to find it here at the
bottom.” He suggested putting it next to the “Search” box at the top of the page.
Participant responses on how easy or difficult it was to find the glossary did not always match
what was observed. For example, a caregiver suggested that the glossary would be in the
“Guidelines for Adults and Adolescents” section, but since the document is available only in
English, he couldn’t look to see. He then suggested going to “Vaccines or Health
Topics/Resources.” He stated that he might be interested in going there because this is a health
topic. He then suggested that he could look under “More Information” in “Search the Glossary.”
However, responding to a question concerning the ease or difficulty of finding the term,
he stated, “It wasn’t so hard.”
29
Once they got to the “Glossary” page, almost all participants typed the acronym “ITIN” (NRTI)
into the “Search” box.
2
However, not everyone saw the search results right away. Because the
page with the search results looks exactly like the entry page, one participant did not realize that
she needed to scroll down to see results of her search. She simply clicked “Search” again and
stared at the screen.
After reading the definition that came up, one patient looked at the screen and commented,
“Maybe it’s better in English,” and clicked on the “Inglés” tab. After reading it in English,
he said simply, “Maybe not.” He then asked the moderator, “What does this mean? Neither in
English nor Spanish. What does this mean? As a person/patient, I want to know what this means.
It’s very dictionary.” A health educator stated that the information was not very useful. “I only
understood that it was a class of information. I would like to see [it] in [a] language that is very
simple, what this does in the body of a person with HIV. ‘This type of med will make it so the
virus makes more and more in your body, or something very simple.’”
Responses regarding the choice of the word “glosario” (glossary) were mixed. While some
participants stated that it was an appropriate term for a medical site, others preferred other words
or phrases, such as “diccionario” (dictionary), which were considered to be more
understandable. Although several participants said that the meaning of “glosario” is clear, a few
of these same participants skipped over the entry, even when reading the titles of the links out
loud. One man said he thought the word chosen should be simple, and suggested the phrase,
“Búsqueda de Términos” (Terms Search). Another recommended “Buscador por Palabras”
(Word Search).
After completing a search and reading the resulting definition, participants rated the
understandability of the information on the page on a scale of 1 (difficult to understand) to
10 (very easy to understand).
Rating Comments
4
Because it’s extremely…I think it’s something that a docto
r would understand. As a
patient I don’t understand what this means. What’s “medicamentosa carente”? I have
an idea, but not really.
6 A little complicated—the steps to get there, the term “glossa
ry.”
8 It helps me.
8
Because although I found the letter, I had to scroll down to find it. [It] took time.
The doctors or
Nurse Practitioners don’t have a lot of time. Needs to be faster and
more direct.
10
Yes, easy to find. Right word to use. [Note that this participant did not find the glossary
until she
was led to it by the facilitator.]
10 Because it was easy and I didn’t have to look for it somepla
ce else.
10
It’s very scientific but understandable. Understandabl
e in that this scientific information
can be difficult to translate.
One of the patients summed up his assessment of the glossary with this statement: “It was useful
that I could enter the word in the “Search” box and it would call it up, but the usefulness ended
2
ITIN stands for inhibidor de la transcriptasa inversa análogo de los nucleótidos.
30
there. The definition was not helpful. I would like to see a definition using words people would
use daily.”
Scenario 5: Live Help and Contact Information
You would like one-on-one assistance about HIV/AIDS information via the internet and/or telephone.
Where would you find this information?
Almost all participants easily found the “Live Help” section of the Web site without prompting.
Most participants went to the “Live Help” section by clicking on the link in the “More
Information” section of the home page. Only a few clicked on the “Live Help” link in the upper
right-hand corner of the home page. Participants’ choice of the link in the “More Information”
section may have been influenced by the fact that they had just read all of the links in
that section.
Most users said they thought the title “Live Help” indicated that they would be interacting with
someone in real time—either by phone or e-mail or live chat. Several said the name made them
think more about a telephone interaction than a computer chat. Following are some of their
comments:
• I would suppose “en vivo” means immediately.
• Probably on the telephone.
• That someone will communicate directly with me. I will hear them or connect with them
somehow.
• When I read “Ayuda en Vivo,” I wasn’t thinking about the internet. I was thinking that I
would talk to someone on the telephone.…What I thought was that I would talk with
someone and they would give me a number.
• It makes me think of telephone or chatting…talking to someone through the internet of
something.
• Access to a person to help you.
In contrast, finding the toll-free number was difficult for most people. Only one person found the
number listed on the top navigation bar. Most users continued to look for the number in the
“Live Help” page. Participants frequently made comments such as “They should have it
someplace obvious!” and stated that they would prefer to see contact information (including
e-mail and a toll-free number) at the top of the “Live Help” page. Several suggested putting it at
the top of the page alongside the hours for “Live Help”:
• When you click on “Live Help,” you expect to see the most frequently asked questions up
front. I’d put the phone number up at the top.
• Like here, the phone number is at the bottom. It should be less hidden.
31
• They should have the phone number here. Sometimes I miss this because I don’t scroll all
the way to the bottom.
The intended audience for the “Live Help” page was unclear to one of the doctors. “Is this page
for professionals or for patients?” he asked. “I thought this was for professionals. But it talks
about confidentiality and has the disclaimer.” This physician also pointed out that most people
are working during the operating hours for the Live Chat (12:00 p.m. to 4:00 p.m. EST). “I don’t
know if it would be better if it were an hour when people are at home or could be looking at the
page outside of the office hours.”
Participants rated the understandability and usefulness of the information on the page on a scale
of 1 (difficult to understand/not useful) to 10 (very easy to understand/very useful).
Rating Comments
5
Because “Liv
e Help” isn’t just chatting. It can be by telephone. If I want this information,
I would like to hear someone’s voice. I would think it would be warmer.
8
Really, it’s good, but it was hard to find the number th
ere at the bottom. Normally you
shouldn’t have to look for it.
8 Easy to understand. It helps. It connects me.
8–9
Because I didn
’t understand who this was for. I wasted time reading this trying to figure
it out.
9
I don’t like this form. Sometimes when one is recently diagnosed and you don’t have
the access or the education it can be inaccessible.…I would like to call and hear
someone’s voice.
9–10 It’s direct. They have information but they have this button I can pu
sh for “Live Help.”
10 Very clear.
10 Simple, clear.
Mobile Formats
The facilitator asked the participants whether they used a smartphone device, such as a
BlackBerry, an iPhone, or an Android phone. If participants did use a smartphone (five of them
did), the facilitator then initiated a discussion on what information from the infoSIDA Web site
would be most helpful to include in a mobile application.
Although participants stated that it would be useful to have the infoSIDA site available in a
mobile format, the testing did not provide any consensus as to which sections would be most
useful to have in that mobile format. One of the physicians interviewed stated that he used his
smartphone for medicines, interactions, doses, and side effects, but not so much for protocols.
Other desires for the mobile application include the following:
• The whole site.
• Just about everything. It’s all important.
• Chat section or messages; clinical trials.
32
• More about HIV and treatment. But I’m not interested in the meds. Vaccines, if they have
discovered anything.
• All of this information is interesting, really. Call attention to prevention, where to get the
test, how it’s transmitted, treated, etc.
• Maybe have a prevention campaign for cell phones. Or have a promotion that talks about
what infoSIDA is.
• I think the order should be prevention, then treatment, then more information. People are
more interested in prevention than anything. And maybe clinical trials.
• Guidelines—because it’s what I use most when I’m with a patient. If I need to know
something right away, I can open the Web site and check. And vaccines. And the
Drug Database.
A patient said that it would be difficult to decide what to put on a cell phone because he does not
know what the goal of the Web site is. “Provide information? Support? On my cell, I would
access the ‘Live Help’—to increase my support network, but information I would probably
access at home.”
Temas de Salud (Health Topics)
None of the participants explored the “Health Topics” section of the Web site during the
completion of the scenarios. Therefore, the facilitator asked about “Health Topics” in
approximately half of the interview sessions.
Participants said they would likely not look in the
section for information because the title “Health
Topics” is so general. “Health Topics” does not
indicate anything related to HIV, and it does not
suggest what information would be included in the
section. One woman gestured to the picture
accompanying the section and stated, “It’s talking
about mothers.”
Participants stated that the type of information they
would expect to find there includes the following:
• Descriptions of HIV
• Where to get the test
• Prevention
• Information about what will to happen with their health and how it will be affected
in general
• Information about how to care for oneself now
• Information about what they should eat
33
• Information about all of the things that are related to HIV, but not necessarily HIV
(substance abuse, cholesterol, depression)
When the interviewer asked participants to open the section, they did like the content featured,
particularly the nutrition information and the section on cultural competence. Many participants
would like to see more information on prevention. A nurse mentioned the “Health Topics”
section lacks information on other STDs; she felt that such information would best fit in the
“Health Topics” section. A physician suggested including information on alternative therapies
such as massage related to HIV: “Things approved or at least it can give you the option,” as well
as community events.
One patient recommended putting a reference to the “Live Help” option under the “infoSIDA
Resources” header on the “Health Topics” page. Another patient suggested renaming the section
as “Resources.” “I see it more as ‘Resources’ than ‘Health Topics.’ That’s so broad. It takes me
out of the topic of HIV—it could be talking about so many things.”
Hojas (Sheets) Versus Pautas (Guidelines)
Most participants said they had no way to know which links (for guidelines and factsheets) were
for professionals and which were for a general audience. However, some participants deduced
that the guidelines were for the providers. One nurse stated, “Upon saying ‘pautas,’ you know
it’s for professionals. A patient can’t take action with this information. It’s understood that this is
for professionals.”
The larger issue seemed to be that while the guidelines are most likely for professionals, the
audience for the factsheets is undefined. One of the participants stated that the word “pautas”
suggests that the content is for professionals, but that “hojas” could be for professionals as well.
Someone else explained, “Yes, ‘pautas’ is probably for professionals. “Hojas” doesn’t really tell
me that it’s for patients, though. It should have a different name. But which [name] would be
difficult to say. I just know that this one isn’t working and it needs a new one.” One of the
doctors stated that the factsheets are “probably to give to the patient about the medicine for HIV.
It doesn’t give me the idea that it would give me as the doctor information. Maybe side effects.”
Participants suggested adding “Recommended for Patients” and “Recommended for
Professionals” or “Information Sheets for Patients” on each of the links, or put the information
intended for professionals in another section.
Even after being shown the two documents, one caregiver still had some difficulty distinguishing
their respective purposes. He posited that the ones titled “Hojas” were brochures that you could
print, and the guidelines were more based on internet or e-mail. He then stated that the “Hojas”
section has simpler language, but that it was not possible to know that from the title.
However, one of the patients saw a large difference in the tone and content of the materials on
different parts of the Web site. Referring to the drug information sheet for Atripla, he
commented, “The patient information seems very technical. As a patient, I don’t feel as though I
can identify with this. I think it’s more for doctors than patients. I don’t know if someone
34
recently diagnosed…I don’t know if this would be the most appropriate. I think this could
confuse a person recently diagnosed.”
However, when given the opportunity to review a patient education handout, the same
participant stated that the sheet was very easy to understand. He liked the way the terms were
defined in a box and he stated that the meanings were “very, very understandable.” He rated the
sheet as “excellent.” “As a patient, I would feel very comfortable giving this to an HIV-positive
person. I’d rate it a 10.…The first part where it says, ‘I’m HIV-positive,’ it’s very personal. I can
identify. The definitions are very simple.”
General Comments
Audience
Some participants were unclear on the audience for and the goal of the infoSIDA Web site.
They questioned whether the site was for doctors or for patients. One patient explained, “I don’t
know…seeing the Web site, I don’t know what the goal of the Web site is. Provide information?
Support?”
Choice of Wording
The same patient also commented that he disliked some of
the wording used on the Web site. “I don’t like to see the
word ‘infectados’ [infected],” he said. He went on to
explain that he prefers the phrase, “person living with
HIV.” (The word “infectados” is used on the home page in
the information written under the “Clinical Trials” section.)
“It’s a reality, we’re infected, but it’s very crude.
HIV-positive, negative, living with HIV. [The word
‘infectados’]…it’s like an ax.”
RSS and Social Media
Although it was not a part of the interview guide and the interviewer did not ask about it, three
participants saw the “RSS” on “Live Help” page and commented that they did not know what the
RSS feature was.
One of the participants suggested linking to social media: “I’d like to see a link to Facebook.
You could post updates on what is happening with medicines, etc.”
35
Final Recommendations
The final recommendations are based on the findings discussed earlier.
• Remove the small infoSIDA logo from the top left-hand side of the home page.
Unless program managers feel strongly that the infoSIDA logo should be at the top, it
may be best to remove it to reduce clutter. Participants did not use it to go to the Spanish-
language site, preferring to click on the “Español” button at the top right-hand side of
the page.
• Change the link for section header “Tratamiento del VIH/SIDA” to take users to a
landing page with the factsheets listed prominently, rather than the current link which
takes users directly to the “Guidelines” page.
• Change the subheader for the factsheets to something more fully descriptive of the
contents. This information should include the intended audience. One suggested title is
“HIV/AIDS Factsheets.”
• On the landing page for the factsheets, either present a bulleted list of the titles of the
factsheets on the center of the page (preferred) or add instructions at the top of the page
directing the reader to the titles listed on the left navigation bar/menu on the left side of
the page.
• Include an “Inglés” tag with the “English” tag after each factsheet or indicate in the
introduction to the page that factsheets are also available in English. Provide a general
link there to remove confusion as to whether the factsheets listed are in English
or Spanish.
• Consider changing the translation of the word “guidelines” to “guías” or “normas.”
• Recommendations: Remove the icons of guidelines covers from all the guidelines pages
and provide a bulleted list with the full name of the guidelines and an “Only in English”
indicator. On pages where the Spanish information cannot be listed first, make all
Spanish-language documentation clearly visible without forcing the user to scroll down
the page.
• List clickable categories of information sheets in the main text of the
“Hojas Informativas” page, rather than placing them in the left navigation bar.
• Either combine or provide a clear explanation for the differences among the three drug
databases listed under the “Drugs” section on the home page.
• On the main “Drug Database” page, move the “Ayuda” (Help) link to be under the term
it is helping to define. Replace the “Ayuda” link with the “Search” link.
• Change the location of the “Search” button on the “infoSIDA Drug Database” page,
placing it immediately after the “Search” box. Move the “Help” button to be either next
to or below the category name for which it is listed (for example, “Drug Class”).
• Relocate “Glossary” to be the first or second header under the “More Information”
section, or add it as a tab on the page. It may be helpful to do both.
36
• Reword the subheader/link to the glossary to read simply “Glosario.” Currently, the
glossary may be difficult to find because a term rather than a word is used to describe the
link. Some of the participants may have overlooked the link because the leading word is
“Búsqueda” (Search) rather than “Glossary.”
• To entice more people to sign up for RSS feeds, include a “What is RSS?” button to
explain this option.
• Because the audience for infoSIDA is so different from the audience for AIDSinfo, it may
be helpful for the “About infoSIDA” link to appear more prominently on the infoSIDA
home page. Currently, it is listed in the footer of the site and is a direct translation of the
“About AIDSinfo” page. Given the feedback from the usability study, this informative
page needs to explain how infoSIDA has been tailored to meet the information needs of
the Spanish-speaking consumer audience.
37
Appendix A: Participant Screeners
Screening Questionnaire: AIDSinfo
Scheduled Date & Time: , @
Directions Sent: Recruiter: Reminder Call:
Participant Screener for Health Care Providers
Usability Test of AIDSinfo & infoSIDA
December 20 & 21, 2010 and January 3, 2011
Location: Baltimore, MD: Baltimore Research Facility
Duration: 90 minutes
Incentive: $250–$350 (depending on medical specialty)
General Recruiting Criteria:
• 9 participants total.
• Must not have participated in a usability test within last 6 months.
• Recruit a mix of English speaking health care professionals using the following breakdown:
4 physicians with at least 50 HIV/AIDS patients
2 physicians with fewer than 50 HIV/AIDS patients with specialty in internal medicine,
infectious disease, family medicine, OB/GYN
2 pharmacists
1 nurse practitioner or physician assistant
General Notes:
• Participants will receive refreshments.
• If participants participate in the study in Baltimore, they will receive incentives of $250 to $350
depending on their medical specialty:
o Infectious disease physician: $350
o Internal medicine or family practice physician: $275
o OB/GYN physician: $350
o Retail-based pharmacist (Pharm.D.): $250
o Hospital-based pharmacist (Pharm.D.): $275
o Nurse practitioner: $250
o Physician assistant: $250
• If participants participate in the study remotely from an alternative location where ICF staff must
connect to their computers, they will receive incentives of $225 to $325 depending on their
medical specialty ($25 less for each medical specialty):
o Infectious disease physician: $325
o Internal medicine or family practice physician: $250
o OB/GYN physician: $325
o Retail-based pharmacist (Pharm.D.): $225
o Hospital-based pharmacist (Pharm.D.): $250
o Nurse practitioner: $225
o Physician assistant: $225
• Screen interactions, audio conversation, and a view of participant’s face will be recorded to hard
drive during testing (recordings should be burned to DVD for the moderator, ideally each testing
day but certainly on the final testing day).
• Participants’ identity and contact information should remain confidential except as needed:
o First names to interact comfortably with participants
A-1
o Phone number if moderator needs to call delayed participants when facility is closed or to
call and remind participant of testing appointment
o Phone number if testing remotely by telephone rather than in person
o E-mail address if testing remotely and must send e-mail link to connect two computers
o Mailing address if testing remotely to mail incentive check.
• Usability testing will be observed live by staff from ICF and NLM.
• Although participants who do not give a lot of feedback will not be penalized in any way, we
encourage recruiting of willing talkers; participants will be encouraged to participate actively by
thinking aloud while exploring the Web site.
Interviewer/Recruiter Introduction
Hello, my name is [recruiter name]; I am calling from Baltimore Research. We are recruiting health care
professionals to participate in usability testing on behalf of the National Library of Medicine. The
purpose of this usability testing is to evaluate a Web site that provides health care professionals,
researchers, people affected by HIV/AIDS, and the general public with access to the U.S. medical
practice guidelines, HIV treatment and prevention clinical trials, and other research information about
HIV/AIDS.
Your participation will help determine what features and enhancements will improve the function of the
Web site.
If you are selected and agree to participate, you will be paid from $225 to $350 for 90 minutes
of your time. I will be able to let you know how much your incentive will be at the end of the
screening interview.
The groups will be conducted on 12/1, 12/20, and 12/21 in Baltimore or remotely using teleconference
and remote meeting software. Would you be interested in participating?
No (thank respondent politely and end call.)
Yes (continue)
Qualifying Questions
I just need to ask some questions to confirm that you qualify for this study.
1. What is your profession?
Physician
• What language do you feel most comfortable speaking?
English (continue to Question 2)
Spanish (skip to termination script)
Both English and Spanish equally (continue to Question 2)
Other language (skip to termination script)
Physician Assistant or Nurse Practitioner
• What language do you feel most comfortable speaking?
English (recruit 1; continue to Question 3)
Spanish (skip to termination script)
Both English and Spanish equally (continue to Question 2)
Other language (skip to termination script)
A-2
Pharmacist (recruit 2 English-speaking pharmacists)
Other (skip to termination script)
2. (If physician by profession, ask,)
Thinking about your current patient load, do you treat people that have any of the following
conditions (keep in mind, you do not necessarily have to treat your patients for these conditions; the
conditions can be a comorbidity to something you treat)
Diabetes
Asthma
High Blood Pressure
HIV/AIDS (Must be checked to continue. Continue to Quextion 2b)
Pregnancy
Cholesterol
Depression
2b. How many HIV/AIDS patients are currently in your case load?
50 or more (recruit 4)
Fewer than 50 (continue to question below)
• What is your specialty area? (recruit 2 total from the following specialties)
Internal medicine (continue)
Infectious disease (continue)
Family medicine (continue)
OB/GYN (continue)
General medicine (continue)
Other (skip to termination script)
3. Gender (ask only if unable to determine; recruit a mix; participant is free to specify transgender)
Male
Female
4. I will read a list of age ranges; please stop me when I reach the range that matches your age (recruit
a mix)
Under 18 (skip to termination script)
18 to 25 (continue)
26 to 35 (continue)
36 to 45 (continue)
46 to 55 (continue)
56 to 65 (continue)
over 65 (continue)
5. When was the last time you participated in a usability test?
(If the participant is not sure what a usability test is, provide a definition. Usability tests involve
professionals in a field or members of the public trying out a product or service to see if it is easy to
use and providing feedback to a researcher or research company.)
Never (continue)
Within the last 3 months (skip to termination script)
Within the last 6 months (continue)
A-3
Within the last 9 months (continue)
Within the last 12 months or longer (continue)
6. How recently did you begin using the Internet?
Within last 6 months (continue)
Within last year (continue)
Longer than 1 year ago (continue)
Do not use the Internet (skip to termination script)
7. How many hours per week do you use the Internet?
Less than 1 hour (skip to termination script)
Between 1 and 5 hours (skip to termination script if began using in last 6 months)
Between 5 and 10 hours (continue)
More than 10 hours (continue)
8. When did you last look for health information on the Internet?
In the last 3 months (continue)
In the last 6 months (wait list)
In the last year (skip to termination script)
Between 1 and 2 years ago (skip to termination script)
More than 2 years ago (skip to termination script)
9. You will most likely need to come to Baltimore to participate. Is this going to be possible?
Yes (skip to invitation section)
No (wait list unless it has been otherwise impossible to fill quota, in which case, offer remote
testing option in Question 10:)
10. (Ask only if unable to recruit enough participants in person)
We may be able to work with you over the Internet while you are at home. Do you have a DSL,
cable, or other high-speed Internet connection at home or at work you could use?
Yes
No (skip to termination script)
(For remote testing we will need to schedule a 5-minute premeeting with them to test the remote
connection at the computer they will be using on date of testing PRIOR to the date scheduled to do
usability testing.)
Termination Script
Unfortunately we already have enough participants who have provided the same answers you have, so
we aren’t able to invite you to participate in this usability test. Thanks for taking the time to answer my
questions. (end call, do not recontact)
Invitation
Thank you for answering my questions. Based on your response, we would like to invite you to
participate in our Web site usability study.
A-4
When participating in this study, your interactions on the screen, your voice, and a camera view of your
face will be video recorded for research purposes; your name will not be associated with the recording
or any of the data collected. Only the research team will have access to the recording, and the recording
will be used only for research purposes. You will be asked to sign a form stating you consent to
participate in the study and to the recording.
The study is focused on assessing the usability and value of a Web site. Questions about your own
background will be limited to Web site-related issues such as your past familiarity with or use of the
Web site.
When scheduling a good time to participate in this study, please consider that you will need 90 minutes
to participate.
Testing Schedule
Are you available to participate for 90 minutes in this study on any of the following dates and times?
(This table will reflect suitable dates and times confirmed with moderator, notetaker, client observers,
and facility staff. If a series of multiple times or dates is open, describe using general language like
“October 6 and 7,” “in the afternoon,” “all day,” etc.)
Date Time Location Participant (open
if empty)
Notes (waiting to
confirm, etc.)
TBD TBD TBD
(If no): I’m sorry we were unable to find a good time for you to participate in this study, but I
appreciate your taking the time to answer my questions. If your schedule changes so that you may be
able to participate, please call me at [number], as we may still have appointments available. Thanks
again for your help. (end call)
14. Can I schedule you for a date and time now?
Yes (find and fill slot in schedule and continue)
No, need to find a good time
That’s fine. When you’ve had a chance to check your schedule, please call me at [number] so
we can be certain to get you an appointment. Thanks again for your help. (end call; call back if
not responded within 24 to 48 hours)
No, not interested
I understand you are not interested in participating, and I appreciate your taking the time to
answer my questions. (end call; do not recontact)
I’m glad that you will be able to join us. At this point I would like to confirm the contact information we
have on file for you.
A-5
Participant Name:
Daytime Phone #
Evening Phone #
Mailing Address (needed only for remote testing to mail incentives check):
E-mail (needed only for remote testing to send link to connect remotely to participant’s computer)
We need 12 people to participate in the testing, so it is very important that you notify us as soon as
possible if for some reason you are unable to participate. Please call or e-mail (insert contact and phone,
e-mail) if this should happen. We look forward to having you participate on (insert appropriate date).
Thank you for your time and we will be in touch again the day before the session.
A-6
Screening Questionnaire – infoSIDA (Spanish)
Encuesta infoSIDA
Fecha y Hora Programada:
Direcciones enviadas:
Reclutador: Recordatorio:
Entrevistador para los pacientes hispa
nohablantes, cuidadores de pacientes,
educadores, trabajadores comunitarios (Omar)
Prueba de Usabilidad de infoSIDA
[Fechas de las pruebas]
6 participantes requeridos (reclutar 9)
Lugar: [a ser determinado más tarde]
Duración: 90 Minutos Incentivo de $100
(Esta entrevista será traducida al español y leída por completo en español.)
Criterio General al Reclutar
• 6 participantes requeridos (Reclutar 9).
• No haber participado en una prueba de usabilidad o grupo de enfoque en los últimos 6 meses.
• Participantes deberán tener la habilidad de leer y conversar en español. La sesión entera se
realizará en español y se solicitará a los participantes a leer material escrito en español.
• Categorías:
o 2 pacientes requeridos (Reclutar 3)
o 2 cuidadores de pacientes requeridos (Reclutar 3)
o 2 educadores en HIV/AIDS o trabajadores comunitarios requeridos (
Reclutar a 3).
Notas Generales:
• Participantes deberán recibir refrigerios.
• Participantes recibirán un incentivo de $100 por 90 minutos de su tiempo.
• Entrevistas, conversaciones y una vista de la cara del participante serán grabadas por
computadora durante las pruebas (grabaciones serán convertidas a DVD para el moderador,
idealmente a diario, pero con seguridad al llegar el día final de las pruebas).
• La identidad y los datos demográficos del participante permanecerán en confidencia a menos que
resulte necesario revelarla:
o Primer nombre para así interactuar cómodamente con los participantes
o Número telefónico por si el moderador necesita llamar a los participantes demorados, si
el local está cerrado, o para llamar y recordar al participante de una cita para una prueba.
• La prueba de usabilidad será observada en vivo por el personal de ICF y NLM.
• Aunque los participantes que no ofrezcan muchas respuestas no serán penalizados en ninguna
forma, animamos a que se recluten personas dispuestas a hablar; participantes serán animados a
participar activamente por medio del razonamiento en voz alta mientras exploran el sitio del Web.
Presentando al entrevistador/reclutador
Hola, mi nombre es Omar Reyes; estoy llamando de parte de ICF Internacional. Estamos reclutando
pacientes, personas cuidadoras de pacientes, y trabajadores comunitarios para participar en una prueba
de usabilidad a nombre de la Biblioteca Nacional de Medicina. El propósito de esta prueba de usabilidad
es el de evaluar un sitio Web que provea a profesionales de la salud, investigadores, personas afectadas
A-7
por el VIH/SIDA, y al público en general, con acceso a las normas de las prácticas médicas de los Estados
Unidos, estudios clínicos para el tratamiento y la prevención del VIH, e información de estudios de
investigación acerca del VIH/SIDA.
Su participación ayudará a determinar cuáles características y realces mejorarán la función del sitio
Web.
Si usted es seleccionado(a) y accede a participar, recibirá $100 por 90 minutos de su tiempo.
Las sesiones en grupo se llevarán a cabo los días lunes 8, miércoles10, y jueves 11 de noviembre del
2010 en Rockville. ¿Estaría usted interesado(a) en participar?
No (Déle las gracias con cortesía y cuelgue.)
Sí (continúe)
Preguntas Calificadoras
Solamente necesito hacer algunas preguntas para confirmar que usted califica para este estudio.
Género (pregunte solamente si no puede determinarlo; reclute una buena mezcla; participante puede
declararse como transgénero)
Masculino Femenino
1. A continuación leeré una gama de edades; por favor deténgame cuando mencione la suya (reclute
una variedad de participantes)
Menos de 18 (prosiga a las instrucciones para terminar la entrevista)
18 a 25 (continúe)
26 a 35 (continúe)
36 a 45 (continúe)
46 a 55 (continúe)
56 a 65 (continúe)
mayor de 65 (continúe)
2. ¿Cuándo fue la última vez que participó en un grupo de enfoque o en una prueba de usabilidad?
(Si el participante no está seguro de lo que es un grupo de enfoque o una prueba de usabilidad, por
favor explíquele. Un grupo de enfoque involucra miembros del público que se reúnen con un
investigador para proveer sus opiniones sobre ciertos productos o servicios. Las pruebas de
usabilidad involucran miembros del público probando un producto o servicio para ver si es fácil de
usar y proveer datos informativos a un investigador o compañía investigativa.)
Nunca (continúe)
En los últimos 3 meses (prosiga a las instrucciones para terminar la entrevista)
Hace más de 5 meses (continúe)
En los últimos 9 meses (continúe)
En los últimos 12 meses o más (continúe)
3. ¿Cuál es su idioma de preferencia?
Solamente inglés (prosiga a las instrucciones para ter
minar la entrevista)
Solamente español (continúe)
Igualmente inglés y español (continúe)
Otro idioma (prosiga a las instrucciones para terminar la entrevista)
A-8
(Esta pregunta se usará para determinar la mezcla de hablantes monolingües y bilingües.)
4. ¿Se siente cómodo leyendo en español?
Sí (continúe)
¿Puede hacer algunas de las siguientes actividades? (Si indica no a todo, prosiga a las
instrucciones para terminar la entrevista)
Lee libros en español (continúe)
Lee periódicos o revistas en español (continúe)
Lee sitios del Web en español (continúe)
No (prosiga a las instrucciones para terminar la entrevista)
5. Favor indique si algunas de las siguientes declaraciones reflejan su familiaridad con el VIH:
(Reclute 3 de las primeras 2 categorías)
Como parte de mi trabajo, educo a personas acerca del VIH – soy un educador de salud
(continúe).
Como parte de mi trabajo, yo ayudo a personas infectadas con el VIH dentro de mi comunidad a
obtener servicios – soy un trabajador comunitario (continúe).
Un amigo o miembro de mi familia ha sido diagnosticado con VIH o SIDA (reclute a 3
específicamente de esta categoría) (continúe)
Yo mismo he sido encontrado positivo para el VIH. (Reclute a 3 específicamente de esta
categoría) (continúe).
Ninguno de los anteriormente mencionados (prosiga a las instrucciones para terminar la
entrevista).
(Las respuestas del investigador para todos aquellos que han seleccionado “Yo mismo” o “Un amigo
o miembro de familia”, deberán permanecer confidenciales y en forma encripto en la computadora
de Omar. Al imprimir, enviar por correo electrónico, o programando horarios, se tendrá mucho
cuidado de no usar apellidos o identificar a los pacientes o a sus cuidadores. Proveedores de
cuidados (pero no de salud) serán identificados simplemente como No-PCS .”)
6. ¿Tiene usted acceso a una computadora ya sea en su hogar o en algún otro lugar?
Sí (continúe)
No (prosiga a las instrucciones para terminar la entrevista)
7. ¿En qué lugar tiene acceso a una computadora? (Puede elegir más de una opción; esta pregunta es
solamente para justificar y confirmar las respuestas para la pregunta número 6)
En casa (continúe)
En el trabajo (continúe)
Local público como biblioteca, escuela, café (continúe)
Otro: (continúe)
8. ¿Puede usar la computadora en cualquier momento?
Sí (continúe)
No (Si no, pregunte cuáles son las restricciones (por ejemplo, miembros de la familia, horario del
trabajo o de la biblioteca, acceso limitado mientras está trabajando) para determinar un nivel de
A-9
familiaridad). Restricciones: (Omar a usar su discreción—si no tienen mucha experiencia
con la computadora, deberá proseguir a las instrucciones para terminar la entrevista).
9. ¿Qué tan cómodo es usted en usar el ratón y el teclado de la computadora?
Extremadamente cómodo (continúe)
Cómodo (continúe)
Un poco cómodo (evite éste si es posible)
No muy cómodo (prosiga a las instrucciones para terminar la entrevista)
Incómodo (prosiga a las instrucciones para terminar la entrevista)
10. ¿Cuán recién ha empezado a usar el Internet?
En los últimos 6 meses (continúe)
En el último año (continúe)
Hace más de 1 año (continúe)
No uso el Internet (prosiga a las instrucciones para terminar la entrevista)
11. ¿Cuántas horas por semana usa el Internet?
Menos de 1 hora (prosiga a las instrucciones para terminar la entrevista)
Entre 1 y 5 horas (prosiga a las instrucciones para terminar la entrevista)
Entre 5 y 10 horas (continúe)
Más de 10 horas (continúe)
12. ¿Qué actividades típicamente realiza en el Internet?
Uso del Facebook, LinkedIn, u otros sitios sociales
Leer noticias y/o artículos deportivos
Usando programas de investigación para buscar información
Comprando a través del Internet en sitios como Amazon / eBay
Compartiendo fotos, música o videos
Enviando o recibiendo correo electrónico
Participando en foros y discusiones por Internet
Otro:
(Continúe si el participante ha seleccionado por lo menos 2 de las actividades. Prosiga a las
instrucciones para terminar la entrevista si el participante no ha seleccionado actividades
apropiadas. Si el participante solamente seleccionó 1 actividad, inclúyalo en una lista de espera.)
13. ¿Cuándo fue la última vez que buscó información acerca de la salud en el Internet?
En los últimos 3 meses (continúe)
En los últimos 6 meses (continúe)
En el ultimo año (coloque en lista de espera; acepte solamente si absolutamente necesita llenar
una cuota)
En los últimos 2 años (prosiga a las instrucciones para terminar la entrevista)
Hace más de 2 años (prosiga a las instrucciones para terminar la entrevista)
14. A como lo mencioné anteriormente, necesitará acudir a Rockville para participar. ¿Será posible
esto?
Sí (prosiga hasta llegar a la sección de la invitación)
No (termine)
Entiendo que usted no puede participar y aprecio el tiempo que ha tomado para
contestar mis preguntas. (Cuelgue el teléfono; no llamar de nuevo)
A-10
Diálogo a llevarse a cabo para terminar la entrevista
Desafortunadamente ya tenemos suficientes participantes quienes han proveído las mismas
respuestas que usted, así es que no podemos invitarlo(a) a participar en esta prueba de usabilidad.
Gracias por tomar el tiempo para contestar mis preguntas. (Cuelgue el teléfono; no llamar de nuevo)
Invitación
Gracias por contestar mis preguntas. Basándonos en sus respuestas, nos gustaría invitarlo(a) a participar
en nuestro estudio de usabilidad por medio de nuestro sitio del Web.
Al participar en este studio, sus interacciones sobre la pantalla, su voz, y una imagen de su rostro será
grabado para el propósito de investigación; su nombre no será asociado con las grabaciones ni con
cualquiera de los datos recolectados. Solamente el equipo de investigación tendrá acceso a las
grabaciones, y las grabaciones serán utilizadas solamente para propósitos investigativos. Se le pedirá a
usted firmar un formulario declarando su consentimiento a participar en el estudio y las grabaciones.
El enfoque del studio es evaluar la usabilidad y el valor del sitio del Web. Las preguntas acerca de su
historial se limitarán a asuntos relacionados con el sitio del Web como por ejemplo asuntos como su
familiarización anterior o uso actual del sitio del Web.
Al programar una hora adecuada para participar en este studio, por favor considere que necesitará 90
minutos para hacerlo.
Horario para las pruebas
¿Está usted disponible a participar en cualquiera de las fechas y horas que se mencionan a
continuación? (Esta tabla reflejará fechas y horas deseables y confirmadas con el moderador, la persona
tomando notas, clientes observadores, y el personal del local. Si se hace disponible una serie múltiple de
horas y fechas, describa usando lenguaje general como “octubre 6 y 7”, “por la tarde”, “todo el día”,
etc.)
Fecha Hora Lugar Participante
(abierto si está
en blanco)
Notas
(esperando a
confirmar, etc.)
A Ser Determinado
ASD ASD
(Si indica que no): Siento mucho que no pudimos encontrar un horario conveniente para usted para
participar en este estudio, pero aprecio el que haya tomado el tiempo para contestar mis preguntas. Si
su horario cambiara en el futuro permitiéndole participar, por favor llámeme al [teléfono], ya que pueda
ser que todavía tengamos citas disponibles. Gracias de nuevo por su ayuda. (Cuelgue el teléfono).
A-11
15. ¿Puedo incluirlo(a) en el calendario ahora mismo con una fecha y hora?
Sí (encuentre lugar en el cuadro, inclúyalo(a) y continúe).
No, necesita encontrar una hora apropiada.
Está bien, Cuando haya tenido una oportunidad de revisar su horario por favor llámeme al
[número] para asegurarnos de hacerle una cita. Gracia otra vez por su ayuda. (Cuelgue el
teléfono; llámelo(a) de nuevo si no responde al transcurso de 24 a 48 horas).
No, no está interesado(a) (termine y no vuelva a contactarlo(a)).
Entiendo que usted no está interesado(a) en participar y aprecio el que usted haya
tomado el tiempo para contestar mis preguntas. (Cuelgue; no vuelva a llamar).
Me complace que podrá participar con nosotros. Ahora me gustaría obtener la información para poder
contactarlo(a) y confirmar su cita, enviarle un recordatorio, o poder avisarle de algún cambio o demora.
Nombre del participante: _____
Número de teléfono
durante el día
_____
Número
de teléfono por la tarde/noche
_____
Necesita
mos de seis a nueve personas participando en esta prueba, por eso es de suma importancia que
nos notifique lo más pronto posible si por alguna razón se le hace imposible participar. Por favor llame o
envíe un mensaje por correo electrónico (inserte contacto, número de teléfono, correo electrónico) si
esto sucediera. Nos agrada que participará con nosotros el (inserte fecha apropiada).
Gracias por su tiempo y estaremos en contacto con usted el día antes de la sesión.
A-12
Appendix B: Moderator Guide
Moderator Guide – AIDSinfo
I. Introduction/ Welcome
Good morning/afternoon. My name is <insert first name>, and I will be your
interviewer. I’m so glad you could make it for our interview today.
Let me give you some background about me and what we’ll be doing today. I am
employed by ICF Macro, a research firm which works for a number of different
government clients. I am interested in hearing your opinions about the AIDSinfo Web
site and some of your ideas on how we can improve the site to make it easier for people
to find the information they need.
Ground Rules
Before we begin, I’d like to review some ground rules and general information for today’s
discussion. I’ve found that these items make the discussion go more smoothly.
Disclosures
• You and I will be the only people talking during the interview, but we are not the only ones
involved. As you can see, I have a note-taker who is observing and who will be capturing
your responses to my questions and other input during our discussion. What you have to
say is very important and we want to make sure that it’s reflected in our final report. I have
some colleagues from the National Library of Medicine, our client, behind the glass. They
are very interested in your responses as well. Even though they are listening to the
conversation they will not be participating, so we can just forget that they are there.
• We are recording this session—We have a computer program that records where you click
on the screen, and the program is also capturing an image of your face—This is only for
recording looks of puzzlement or surprise or other facial expressions as you’re searching. It
is NOT for identification.
• Although we will be writing a report about our findings from these series of interviews, you
will not be identified by name in the report…I’m only interested in what is said so I can
accurately describe our discussion.
• The persons observing our discussion today have signed a confidentiality agreement.
Nothing you will say will affect your use of this Web site; and, as I said, your identity will not
be revealed in my report.
There are no right or wrong answers to the questions I ask or the tasks I’ll ask you to
complete. We expect differences in how people use the Web site, and we need to know
about these differences. I’d like to know how you would go about doing things on this site if
no one were watching you.
B-1
Your participation is voluntary. If, at any point in time you feel you need to end the
interview, please let me know.
I am not an area expert in this subject or this Web site. Although I would like to know what
questions you have about the site as we’re going through some scenarios, I’m not in a
position to answer any of your technical questions. This also means that I’m not going to be
offended if you tell me that you don’t like something or you think it’s not organized well. So
please be honest about the Web site we’ll be reviewing without fearing that you are going
to offend me in some way.
We estimate that it will take about 1.5 hours to complete this interview.
Icebreaker
So…to begin this afternoon’s/this evening’s discussion, can you please tell me—what are
your favorite three health Web sites and what do you like best about them?
First Impressions
Today we’re going to be looking at a site called AIDSinfo. (Bring up site).
Have you ever visited this site before?
o (If yes) What has been your experience with it?
Looking at the site now, what are your impressions of it?
o What do you most like about it?
o What do you like least about it?
Tasks
I’m going to ask you to complete several tasks on the Web site. I’ll give you several scenarios to
look at, and I’d like you to complete them the way you would if you were at home or at the
office without anyone watching you. The only difference is that I’m going to ask you to think
out loud—I can see where you move the mouse, but I don’t know what you’re thinking and why
you’re moving the cursor to various areas of the screen unless you tell me.
Please keep in mind that this is NOT a test. We know that different people use the Web site
differently, and we want to find out how to make this site easier and more intuitive for people
to find the information they need. So if you have any suggestions about changes as we go
through these scenarios, feel free to let me know about those, too.
B-2
Scenario 1: Locating the guidelines
Here is the first scenario (pass out scenario 1). Would you please read it out loud?
You have a 22 year-old non-pregnant female patient, newly diagnosed HIV-positive. You need to
find out what HIV regimen to prescribe for her. Your patient also does not know anything about
HIV and needs patient information for herself (e.g., what it means to be infected with HIV).
o Goal: Find the guidelines section of the Web site, specifically the adult and adolescent treatment
guidelines.
Where in the AIDSinfo site would you go to find information on what HIV regimen to
prescribe for her?
Which of these links would you be most likely to choose to find the information you
need?
Where would you click to find information about what HIV regimen to prescribe
her?
If participant is not able to locate the Adult and Adolescent treatment guidelines, direct them to
the information and ask them, what challenges did you encounter?
FOR PDF
o Goal: Find out how professionals use the Guidelines
Here is the document you said you would like to look at. How do you usually find
specific information in a large document such as this one?
Let’s go ahead and look for the information discussed in the scenario. Please talk me
through what you’re finding and how relevant it is to answering the initial question
of what regimen to prescribe for her.
How familiar are you with this feature (point at bookmarks feature)?
If you are given the opportunity to choose between a PDF and an HTML version of a
document, which version do you usually choose?
What are some reasons for your preference?
IF PARTICIPANT SELECTS AN HTML LINK FIRST:
What do you think about the appearance of this page?
If you wanted to search this document, how would you do it?
If you wanted to go to the Table of Contents, how would you do it?
Any suggestions for making this task easier?
Goal: Find guideline derivative products (e.g., patient education materials in particular the
“HIV and its Treatment fact sheet series).
Where in the AIDSinfo site would you go to find information to give to your patient?
Which information would you recommend that she read?
How logical is the placement of the patient fact sheets on the guideline pages
o
Where else would you like to see them?
Any suggestions for making this task easier?
B-3
Scenario 2: Using the Drug Database to Locate Drug Information
Now that you have looked up information on a treatment regimen, let’s find some other
information. (Pass out paper with Scenario 2.) Please read it out loud.
You have determined that the non-pregnant female patient from Scenario 1 needs to take
Atripla which is a fixed dose combination of Efavirenz, Emtricitabine, and Tenofovir DF.
o Goal: Use the professional drug fact sheet to learn about a drug.
Where would you find dosing information so that you know how to write the
prescription?
Where else on the site could you look?
o Goal: Use the lay fact sheet to print and give to the patient as supporting material.
Where on the AIDSinfo site would you find side effect information to give to the
patient?
How will you print the information?
[Have the participant read through portions of the fact sheet, and ask:]
o How appropriate is the level of detail provided for your patient?
o What do you think of the reading level of this fact sheet?
If participant is not able to locate drug information (either the professional or lay fact sheet),
direct them to the information and ask them,
• What challenges did you encounter?
• What made it difficult for you to locate information about Atripla?
o Goal: Assess interconnectivity of content across the sections (using the drug database to
locate relevant trials for a particular drug).
What clinical trial information is available for Atripla?
The user could do one of two things: 1. Scroll to the bottom of the Atripla drug fact sheet (either
the professional or the lay version) and click on the link for trials OR use the clinical trial search
section of the web site. See which one they select and inquire if they could find the other option.
What other ways are there to find that information on this site?
How easy was it to find the clinical trial information?
How useful was the information once you found it?
Any suggestions for making this task easier?
Scenario 3: Using the Clinical Trial Search
Let’s look at a scenario regarding a different patient. (Pass out Scenario 3). START BACK AT
HOME PAGE
B-4
You have an HIV positive patient who is diagnosed with Hepatitis C and wants to participate in a
clinical trial.
o Goal: Search for clinical trials (using basic, advanced, or quick search).
Where on the AIDSinfo Web site would you look for clinical trials related to Hep C?
The user has the option of doing a basic, quick or advanced search. See which one they use and
inquire as to why they choose that option (over the others). [If the user does not use the quick
search, or is not able to find information on clinical trials, direct them to the search link and ask
the questions below.]
• It looks like the web site provides several different ways to search for clinical trials. What are
some of the reasons you chose to use the option you did?
• (If they didn’t look at the “Quick Search” option) What do you think would be under the “Quick
Search” link?
o (Show option) What do you think about it once you look at it?
o How can we make it clearer that the “Quick Search” is a listing of “canned trials?”
On a scale of 1-10, with one being difficult to understand and 10 being very easy to
understand, how easy was it to find the information you were looking for?
What are some of the reasons you gave it that score?
What would you like to see?
Any suggestions for making this task easier?
Your patient will be moving to Baltimore soon and would like to know if there are trials available
near there. How would you find that out?
Once participant locates a listing of Hepatitis C studies, ask the questions below. If participant is not able
to limit the results to show only those studies in Baltimore, direct them to the “refine results” option.
Ask them to use this screen to limit their results so that only studies from Baltimore display. Ask them,
what challenges did you encounter?
o Goal: Refine the results, and find specific contact information for the trial (name, phone number,
email address of a study coordinator).
How can you limit the studies on your computer so that you only see studies specifically in
the Baltimore area?
If you are interested in this study, who would you contact to see if your patient could be
eligible to participate?
How can this page be improved so that it is easier to refine results?
On a scale of 1-10, with one being difficult to understand and 10 being very easy to
understand, how easy was it to refine your search to find the information you
needed?
What are some of the reasons you gave it that score?
What would you like to see?
Any suggestions for making this task easier?
B-5
Scenario 4: Using Health Topics to Locate Relevant Links/Resources
Let’s look another scenario: (Pass out Scenario 4).
Your clinic is currently in need of additional funding to support a new treatment adherence
program that you would like to implement. Where on the AIDSinfo site can you search for
funding opportunities?
o Goal: Find how professionals may use the “Health Topics” sections
START FROM HOME PAGE—if they click on anything related to Health Topics, take them to the Health
Topics tab
On the health topics page (staging site), ask them to briefly read the information on the page
and ask the questions below. If participant is not able to locate health topics or clicks but leaves
the page direct them to the page and ask them, what challenges did you encounter? Why were
you not able to locate information regarding funding opportunities?
If they find via search box, then return to Health Topics tab and probe:
How would you find funding from this page (show Health Topics page)?
On a scale of 1-10, with one being poorly organized and 10 being well organized, how well
organized is the information on this page?
What are some of the reasons you gave it that score?
Any suggestions for making this page better organized?
What do you think of the information contained on this page?
The purpose of this sectionis to provide links to resources (AIDSinfo and non-AIDSinfo)
grouped by topic. How well dooes the current title of the section, “Health Topics” capture
that?
o What are some alternate names you would suggest for this page?
Scenario 5: Mobile Formats
One of the areas we are interested in knowing more about is how providers use their mobile
devices. Do you use a smart phone (Blackberry, iPhone), iPod Touch, or other mobile device?
If yes: How frequently do you access health information on it?
Probe: What are health information sites you typically access?
• What things do you like or dislike about the sites you access on your mobile device?
What makes them easier or harder to use?
• As you can see, the Guidelines documents are fairly large. If we were to create a
shortened version of the Guidelines for mobile devices, which parts of the
Guidelines would be the most important to include?
How have you seen other health information sites deal with large
documents such as the Guidelines?
What do you like or dislike about what they did?
B-6
• How about the drug database?
What other drug information sites do you visit on your mobile device?
What do you like or dislike about their mobile interface?
What information would you want to see included in a mobile-friendly
version of the drug fact sheet you saw here on AIDSinfo?
If we were to create mobile-friendly versions of our drug fact
sheets, how would you like them to look?
How much detail would you want?
• Probe—bulleted information, or the full information equal
to what you just saw on the AIDSinfo website?
Scenario 6: Locating Contact Information for Help
Let’s look at one final scenario (Pass out Scenario 5). Again, please read it out loud.
You would like one-on-one assistance about HIV/AIDS information via the Internet and/or
telephone. Where would you find this information on the AIDSinfo website?
o Goal 1: Find out if participant understands that live chat services are available via Live Help.
o Goal 2: Determine whether: 1) the term “Live Help” is appropriate, 2) participant understands
that Live Help is a chat service, 3) the placement of Live Help and the 1-800 number are where
people would look for help, and 4) participant knows that there are other help methods
available besides Live Help (i.e., phone and email).
On the live help page, ask them to briefly read the information on the page and ask the
questions below. [If participant is not able to locate “live help” direct them to the page and ask
them, what challenges did you encounter? Why were you not able to locate “live help”?]
What do you think about the phrase “live help”?
What does it mean to you? (What kind of help do you think you will
receive?)
How clear is it that this is a live chat option? What could make it clearer?
What do you think of the placement of the 1-800 number?
If you needed assistance after hours and Live Help or the toll free number was not
available, what other options are available to you?
On a scale of 1-10, with one being difficult to understand and 10 being very easy to
understand, how well is this page organized so that you can find the information you need?
What are some of the reasons you gave it that score?
What would you like to see?
Any suggestions for making this task easier?
False Close
As I mentioned earlier, there are some observers watching our discussion. I’m going to step out
of the room for a minute and see if there are any other questions they would like me to ask.
B-7
Wrap-up
Do you have any other comments you would like to share with me about what you like or
dislike about the AIDSinfo website or how you would like to see it improved?
On behalf of the National Library of Medicine, thank you for your time. We really appreciate
your input on how to make this site better. Please be sure to stop by and see _____ on your
way out for payment. Thanks again.
B-8
Moderator Guide: infoSIDA (English)
I. Introduction/ Welcome
Good morning/afternoon. My name is <insert first name>, and I will be your interviewer. I’m so
glad you could make it for our interview today.
Let me give you some background about me and what we’ll be doing today. I am employed by
ICF Macro, a research firm which works for a number of different government clients. I am
interested in hearing your opinions about the infoSIDA Web site and some of your ideas on how
we can improve the site to make it easier for people to find the information they need.
Ground Rules
Before we begin, I’d like to review some ground rules and general information for today’s discussion.
I’ve found that these items make the discussion go more smoothly.
Disclosures
• You and I will be the only people talking during the interview, but we are not the only ones involved.
As you can see, I have a note-taker who is observing and who will be capturing your responses to my
questions and other input during our discussion. What you have to say is very important and we
want to make sure that it’s reflected in our final report. I have some colleagues from the National
Library of Medicine, our client, behind the glass. They are very interested in your responses as well.
Even though they are listening to the conversation they will not be participating, so we can just
forget that they are there.
• We are recording this session—We have a computer program that records where you click on the
screen, and the program is also capturing an image of your face—This is only for recording looks of
puzzlement or surprise or other facial expressions as you’re searching. It is NOT for identification.
• Although we will be writing a report about our findings from these series of interviews, you will not
be identified by name in the report…I’m only interested in what is said so I can accurately describe
our discussion.
• The persons observing our discussion today have signed a confidentiality agreement. Nothing you
will say will affect your use of this Web site; and, as I said, your identity will not be revealed in my
report.
There are no right or wrong answers to the questions I ask or the tasks I’ll ask you to complete. We
expect differences in how people use the Web site, and we need to know about these differences.
I’d like to know how you would go about doing things on this site if no one were watching you.
Your participation is voluntary. If, at any point in time you feel you need to end the interview, please
let me know.
I am not an area expert in this subject or this Web site. Although I would like to know what
questions you have about the site as we’re going through some scenarios, I’m not in a position to
answer any of your technical questions. This also means that I’m not going to be offended if you tell
B-9
me that you don’t like something or you think it’s not organized well. So please be honest about the
Web site we’ll be reviewing without fearing that you are going to offend me in some way.
We estimate that it will take about 1.5 hours to complete this interview.
Icebreaker
So…to begin this afternoon’s/this evening’s discussion, can you please tell me—what are your
favorite three health Web sites and what do you like best about them?
First Impressions
Today we’re going to be looking at a site called AIDSinfo. (Bring up site).
Have you ever visited this site before?
o (If yes) What has been your experience with it?
Looking at the site now, what are your impressions of it?
o What do you most like about it?
o What do you like least about it?
Tasks
I’m going to ask you to complete several tasks on the Web site. I’ll give you several scenarios to look at,
and I’d like you to complete them the way you would if you were at home or at the office without
anyone watching you. The only difference is that I’m going to ask you to think out loud—I can see where
you move the mouse, but I don’t know what you’re thinking and why you’re moving the cursor to
various areas of the screen unless you tell me.
Please keep in mind that this is NOT a test. We know that different people use the Web site differently,
and we want to find out how to make this site easier and more intuitive for people to find the
information they need. So if you have any suggestions about changes as we go through these scenarios,
feel free to let me know about those, too.
B-10
Scenario 1: Locating the patient fact sheet
Here is the first scenario (pass out scenario 1). Would you please read it out loud?
You are looking for information in Spanish about starting HIV medications. You have come to the AIDS
Info Web site to find this information.
o Goal: Participant to locate a fact sheet (specifically the “HIV and its Treatment fact sheet series)
Where in the AIDS info site would you go to find information in Spanish?
What would you do first? Why?
What are you looking for?
What words are going through your mind?
Why did you go to that particular place on the Web site? What would you do next?
Note to facilitator: When searching for patient fact sheets in Spanish, users have the option of
navigating through AIDSinfo in English and finding the equivalent Spanish resource listed on an English
page OR they have the option of navigating through infoSIDA in Spanish. Ask the user why they selected
the path they did. Ask them to briefly read part of the fact sheet and ask the questions below. If
participant is not able to locate fact sheet, direct them to the fact sheet and ask them, what challenges
did you encounter? Why were you not able to find the fact sheet?
o Goal: Participant to read part of the fact sheet, and indicate whether they understand it.
Did you understand the information being provided on the fact sheet? Is it easy to understand?
Is it too complicated? If so, what is complicated?
What would you like to see?
Any suggestions for making this task easier?
Scenario 2: Locating Guidelines (in English)
Let’s move to the next scenario. (Pass out paper with Scenario 2) Please read it out loud.
You want to know the standard of care for when doctors should begin treating HIV positive patients.
o Goal: Find the Guidelines (in English).
What would you do first? Why?
What are you looking for?
What words are going through your mind?
Why did you go to that particular place on the Web site? What would you do next?
Note to facilitator: On the guideline page, ask them to briefly read Web page and ask the questions
below. If participant is not able to find the guidelines direct them to the guidelines and ask them, what
challenges did you encounter? Why were you not able to locate the guidelines?Is the term “pautas”
clear?
o Goal: Find out if participant understands that the guidelines are not available in Spanish. Get some
feedback about the design/layout of the page so that this information can be more transparent to
the user.
B-11
Did you understand the information being provided on this page? On a scale of 1-10, with one
being very difficult to understand and 10 being very easy to understand, how understandable is
the information on this page?
What are some of the reasons you gave it that score?
Did you understand that the Guidelines are only available in English when you came to this
page? In what languages is this information available?
Are all of the materials available (repeat the participant opinion)-in Spanish/in both Spanish and
English/only English?
Which documents are available in Spanish?
What would you like to see that explains to you that they are only available in English?
If you are looking to read Guideline information in Spanish and you realize that they are only in
English, is there anything else in Spanish that can help you answer your question?
Any suggestions for making this task easier?
SESSION 3: Using the Drug Database to Locate Drug Information
Now let’s have you look for some other information. (Pass out scenario 3) Please read it out loud.
You are interested in side effect information for Atripla. What would you do to find this information?
o Goal: have participant find and use the Spanish drug database to locate the Atripla fact sheet.
What would you do first? Why?
What are you looking for?
What words are going through your mind?
Why did you go to that particular place on the Web site? What would you do next?
Note to facilitator: Once participant has clicked on the drug database, ask them:
How would you search for information on Atripla? [The user has 3 options: they can search
manually, use the A-Z index, or use the left-navigation.]
Are there any other ways you can locate information on Atripla? [If the participant is not able to
locate the fact sheet using another retrieval method, show them the other two options and ask
them, what challenges did you encounter? Why were you not able to locate the information?]
o Goal: Assess interconnectivity of content across the sections (using the drug database to locate
relevant trials for a particular drug).
Where on the infoSIDA site would you look for information on clinical trials for Atripla?
Note to facilitator: The user could do one of two things: 1. Scroll to the bottom of the Atripla drug fact
sheet and click on the link for trials OR use the clinical trial search section of the web site. See which one
they select and inquire if they could find the other option.
Did you find the information you were looking for? Yes or NO? Why or why not?
On a scale of 1-10, with one being difficult to understand and 10 being very easy to understand,
how understandable was the information on this page?
B-12
What are some of the reasons you gave it that score?
What would you like to see?
Any suggestions for making this task easier?
Scenario 4: Accessing and Using the Spanish Glossary
let’s look at another scenario: (Pass out scenario 4) please read it out loud.
As you were reading the drug record on Atripla, you read that this pill contains 3 active drugs. Two of the
active drugs are from the NRTI or nucleoside reverse transcriptase inhibitor drug class and you want to
know more about what this word means. Where would you find more information about the word’s
meaning?
o Goal: Find and use the Spanish glossary to learn more about a complicated term.
What would you do first? Why?
What are you looking for?
What words are going through your mind?
Why did you go to that particular place on the Web site? What would you do next?
Note to facilitator: Once participant has found the glossary, ask them to briefly locate the definition for
NRTI and then ask the questions below. [If participant is not able to locate glossary, direct them to the
page and ask them, what challenges did you encounter? Why were you not able to locate the glossary
page?]
On a scale of 1-10, with one being difficult to understand and 10 being very easy to understand,
how understandable was the information on this page?
What are some of the reasons you gave it a (state number)?
How easy or difficult was it to find the glossary?
Is the term “glosario” an appropriate term to use when referring to a dictionary of terms? Why
or why not?
How helpful was the information? What was most helpful? What was least helpful?
What would you like to see?
Any suggestions for making this task easier?
Scenario 5: Locating Contact Information for Help
Let’s look at another scenario. (Pass out scenario 5) please read it out loud.
You would like one-on-one assistance about HIV/AIDS information via the Internet and/or telephone.
Where would you find this information?
o Goal: Find out if participant understands that live chat services are available via Live Help.
What would you do first? Why?
What are you looking for?
What words are going through your mind?
Why did you go to that particular place on the Web site? What would you do next?
B-13
Note to facilitator: On the live help page, ask them to briefly read the information on the page and ask
the questions below. If participant is not able to locate “live help” direct them to the page and ask them,
what challenges did you encounter? Why were you not able to locate the “live help”?
o Goal: Determine whether: 1) the term “Live Help” is appropriate, 2) participant understands that Live
Help is a chat service, 3) the placement of Live Help and the 1-800 number are where people would
look for help, and 4) participant knows that there are other help methods available besides Live Help
(i.e., phone and email).
On a scale of 1-10, with one being difficult to understand and 10 being very easy to understand,
how understandable was the information on this page?
What are some of the reasons you gave it a (state number)?
When you see the phrase, “ayuda en vivo,” what comes to mind?
How easy was it to locate the 1-800 number?
On a scale of 1-10, with one being not at all helpful and 10 being very helpful, how helpful was
the information on this page?
What are some of the reasons you gave it a (state number)?
What would you like to see?
Any suggestions for making this task easier?
Scenario 6: Mobile Formats
One of the areas we are interested in knowing more about is how providers use their mobile devices. Do
you use a smart phone (Blackberry, iPhone), iPod Touch, or other mobile device?
If yes: How frequently do you access health protocol information on it?
Probe: What are health information the sites you typically access?
• What things do you like or dislike about the sites you access on your mobile device?
Probe: What makes them easier or harder to use?
• If we were to release a mobile version of infoSIDA, what parts of the site would you
want to see on a mobile device?
Probe: would the fact sheets, glossary, clinical trials, health topics, specialty
pages, and the translation tool be beneficial to include on a mobile device?
Why or why not?
Have you seen other health information sites provide this service via a
mobile phone? What did you like or dislike about it?
• How about the drug database?
Probe: What other database-driven sites do you visit on your mobile device?
What do you like or dislike about how their mobile interface?
B-14
False Close
As I mentioned earlier, there are some observers watching our discussion. I’m going to step out of the
room for a minute and see if there are any other questions they would like me to ask.
Wrap-up
Do you have any other comments you would like to share with me about what you like or dislike about
the infoSIDA website or how you would like to see it improved?
On behalf of the National Library of Medicine, thank you for your time. We really appreciate your input
on how to make this site better. Please be sure to stop by and see _____ on your way out for payment.
Thanks again.
B-15
Moderator Guide – infoSIDA (Spanish)
Guía para la entrevista de InfoSIDA
I. Presentación/bienvenida
Buenos días/buenas tardes. Me llamo <escriba el nombre> y seré su entrevistador. Me alegra
que estuviera disponible hoy para nuestra entrevista .
Le contaré un poco acerca de quien soy y de lo que haremos hoy. Soy empleado de ICF Macro,
una firma de investigación que trabaja para una variedad de clientes del gobierno. Estoy
interesado(a) en escuchar sus opiniones acerca del sitio Web de infoSIDA y de algunas de sus
ideas sobre cómo podemos mejorar el sitio para facilitarle a las personas encontrar la
información que necesitan.
Reglas básicas
Antes de empezar, me gustaría repasar algunas reglas básicas e información general para el debate de
hoy. He encontrado que estas cosas hacen que el debate marche mejor.
Divulgaciones
• Usted y yo seremos los únicos hablando durante la entrevista, pero no somos los únicos
involucrados. Como puede ver, tenemos una persona tomando notas que está observando, y
capturará sus respuestas a mis preguntas y otros aportes durante nuestro debate. Lo que usted
tiene que decir es muy importante y queremos asegurarnos de que se refleje en nuestro informe
final. Detrás del vidrio, tengo a varios colegas de la Biblioteca Nacional de Medicina que son clientes
nuestros. Ellos también están muy interesados en sus respuestas. Aunque están escuchando la
conversación, no participarán, entonces nos podemos olvidar de que están ahí.
• Estamos grabando esta sesión; tenemos un programa de computadora que graba cuando usted
hace clic en la pantalla y además captura una imagen de su rostro; esto con el fin de capturar el
desconcierto o sorpresa en su cara u otras expresiones faciales a medida que usted hace su
búsqueda. NO es para identificarlo(a).
• Aun cuando estaremos escribiendo un informe acerca de nuestros hallazgos durante esta serie de
entrevistas, su nombre no será identificado en el informe…sólo estoy interesado(a) en lo que se dice
para que pueda describir acertadamente nuestro debate.
• Las personas que observan nuestro debate de hoy han firmado un acuerdo de confidencialidad.
Nada de lo que diga afectará su uso de este sitio Web, y como dije, su identidad no será divulgada
en mi informe.
No hay respuestas correctas o incorrectas a las preguntas que hago o a las tareas que le pediré que
realice. Anticipamos diferencias en la forma como las personas usan el sitio Web y debemos saber
acerca de estas diferencias. Me gustaría saber cómo usted haría su investigación en este sitio si
nadie lo estuviera mirando.
B-16
Su participación es voluntaria. Si en algún momento siente que debe suspender esta entrevista,
dígamelo.
No soy un experto en este tema o en este sitio Web. Aunque me gustaría saber qué preguntas tiene
acerca de este sitio a medida que vamos por los diferentes escenarios, no estoy en posición de
contestar ninguna de sus preguntas técnicas. Esto también significa que no me voy a ofender si me
dice que no le gusta algo o si piensa que no está muy bien organizado. De manera que sea
honesto(a) acerca del sitio Web que vamos a revisar y no piense que me va a ofender de alguna
manera.
Estimamos que esta entrevista tomará aproximadamente 1 hora y media.
Rompehielos
Entonces, para empezar el debate de esta tarde/esta noche, ¿me podría decir cuáles son sus tres
sitios Web favoritos y qué es lo que más le gusta de ellos?
Primeras impresiones
Hoy vamos a estar mirando un sitio llamado AIDSinfo. (Entre al sitio).
¿Ha visitado alguna vez este sitio?
o (si la respuesta es positiva) ¿cuál ha sido su experiencia con el sitio?
Mirando el sitio ahora, ¿cuáles son sus impresiones?
o ¿Qué es lo que más le gusta?
o ¿Qué es lo que menos le gusta?
Tareas
Le voy a pedir que realice varias tareas en el sitio Web. Le daré varios escenarios y me gustaría que los
complete como lo haría si estuviera en su casa o en la oficina sin que nadie lo esté observando. La única
diferencia es que le voy a pedir que piense en voz alta – yo puedo ver dónde coloca el cursor pero no
puedo saber qué está pensando y porqué está moviendo el cursor por diferentes áreas de la pantalla, a
no ser que me lo diga.
Por favor, recuerde que esto NO es un examen. Sabemos que diferentes personas usan el sitio Web de
diferentes maneras, y queremos saber cómo podemos hacer este sitio más fácil y más intuitivo para que
las personas encuentren la información que necesitan. De manera que si tiene algunas sugerencias de
cambios mientras miramos los escenarios, también déjemelo saber, por favor.
Escenario 1: Cómo encontrar la hoja informativa para pacientes
Aquí está el primer escenario (entregar el escenario 1). ¿Podría leerlo en voz alta por favor?
Estamos buscando información en español acerca de empezar los medicamentos contra el VIH. Ha
llegado al sitio Web de AIDSinfo para encontrar esta información.
o Meta: que el participante localice una hoja informativa (específicamente la hoja informativa de la
serie “el VIH y su tratamiento"
B-17
¿A qué parte del sitio de AIDSinfo iría para encontrar información en español?
¿Qué haría primero? ¿Por qué?
¿Qué está buscando?
¿En qué palabras está pensando?
¿Por qué fue a ese lugar específico del sitio Web? ¿Cuál sería el paso siguiente?
Nota para el facilitador: cuando se trata de buscar hojas informativas en español, el usuario tiene la
opción de navegar por AIDSinfo en inglés y encontrar el recurso equivalente en español que aparece en la
página en inglés O tienen la opción de navegar por infoSIDA en español. Pregunte al usuario por qué
seleccionó el camino que escogió. Pídale que lea brevemente parte de la hoja informativa y hágale las
preguntas que aparecen debajo. Si el participante no puede localizar la hoja informativa, diríjalo a la
hoja informativa y pregúntele, ¿qué retos encontró? ¿Por qué no pudo encontrar la hoja informativa?
o Meta: que el participante lea parte de la hoja informativa e indique si la entiende o no.
¿Entendió la información que aparece en la hoja informativa? ¿Se entiende fácilmente?
¿Es muy complicada? En caso afirmativo, ¿qué es lo complicado?
¿Qué le gustaría ver?
¿Alguna sugerencia para facilitar esta tarea?
Escenario 2: Cómo localizar las Pautas (en inglés)
Pasemos al siguiente escenario. (Entregue el papel con el escenario 2). Por favor léalo en voz alta.
Usted desea saber cuál es la norma aceptable de cuándo deben los médicos empezar a tratar a los
pacientes VIH positivos.
o Meta: encontrar las Pautas (en inglés)
¿Qué haría primero? ¿Por qué?
¿Qué está buscando?
¿En qué palabras está pensando?
¿Por qué fue a ese lugar específico del sitio Web? ¿Cuál sería el paso siguiente?
Nota para el facilitador: en la página de las Pautas, pídale que lea brevemente la página Web y hágale
las preguntas que aparecen debajo. Si el participante no puede localizar las Pautas, diríjalo a las Pautas y
pregúntele, ¿qué retos encontró? ¿Por qué no pudo localizar las Pautas? ¿Se entiende la palabra
“pautas”? ¿Hay otra palabra mejor?
o Meta: saber si el participante entiende que las Pautas no están disponibles en español. Pida
retroalimentación acerca del diseño/esquema de la página para que esta información pueda ser más
transparente para el usuario.
¿Entendió la información que aparece en esta página? En una escala del 1 al 10, donde uno es
muy difícil de entender y 10 muy fácil de entender, ¿cómo califica la información de esta
página?
¿Cuáles son algunas de las razones para darle ese puntaje?
B-18
¿Entendió que las Pautas estaban disponibles solamente en inglés cuando llegó a esta página?
¿En qué idiomas está disponible esta información?
¿Están todos los materiales disponibles (repita la opinión del participante) en español/tanto en
español como en inglés/solamente en inglés?
¿Qué documentos están disponibles en español?
¿En qué manera le gustaría saber que están solamente disponibles en inglés?
Si está buscando leer en español información sobre las Pautas y se da cuenta que está
disponible solamente en inglés, ¿hay alguna otra cosa en español que le pudiera ayudar a
contestar su pregunta?
¿Alguna sugerencia para facilitar esta tarea?
Escenario 3: Cómo usar la base de datos de medicamentos para localizar
información sobre un medicamento
Ahora usted va a buscar otra información. (Entregue el papel con el escenario 3). Por favor léalo en voz
alta.
Usted está interesado(a) en encontrar información sobre los efectos secundarios del Atripla. ¿Qué haría
para encontrar esta información?
o Meta: que el participante encuentre y use la base de datos de medicamentos en español para
localizar la hoja informativa del Atripla.
¿Qué haría primero? ¿Por qué?
¿Qué está buscando?
¿En qué palabras está pensando?
¿Por qué fue a ese lugar específico del sitio Web? ¿Cuál sería el paso siguiente?
Nota para el facilitador: una vez que el participante haya hecho clic en la base de datos de
medicamentos, pregúntele:
¿Cómo buscaría la información sobre el Atripla? [El usuario tiene 3 opciones: puede hacer una
búsqueda manual, usar el índice de A-Z ó usar la navegación que aparece a la izquierda.]
¿Hay alguna otra manera de localizar la información sobre el Atripla? [Si el participante no pudo
localizar la hoja informativa usando otro método de búsqueda, muéstrele las otras dos opciones
y pregúntele, ¿qué retos encontró? ¿Por qué no pudo localizar la información?]
o Meta: evaluar la interconectividad del contenido a través de las secciones (usando la base de datos
de medicamentos para localizar ensayos clínicos relevantes a un medicamento específico.
¿En qué parte del sitio de infoSIDA buscaría información sobre los ensayos clínicos del Atripla?
Nota para el facilitador: el usuario podría hacer una de dos cosas: 1. Bajar hasta el final de la hoja
informativa del Atripla y hacer clic en el enlace a los ensayos clínicos Ó, 2. Usar la sección de búsqueda
del sitio Web para ensayos clínicos. Vea cuál selecciona y pregunte si podría encontrar la otra opción.
¿Encontró la información que buscaba? ¿Sí o No? ¿Por qué sí o por qué no?
B-19
En una escala del 1 al 10, donde uno es muy difícil de entender y 10 muy fácil de entender,
¿cómo califica la información de esta página?
¿Cuáles son algunas de las razones para darle ese puntaje?
¿Qué le gustaría ver?
¿Alguna sugerencia para facilitar esta tarea?
Escenario 4: Cómo acceder y usar el glosario en español
Miremos otro escenario: (Entregue el papel con el escenario 4). Por favor léalo en voz alta.
Mientras leía el registro del medicamento Atripla, se encontró con que esta pastilla contiene 3
medicamentos activos. Dos de los medicamentos activos son de la clase de medicamentos ITIN o
inhibidores de la transcriptasa inversa análogos de los nucleósidos, y usted desea saber qué significa esa
palabra. ¿Dónde encontraría más información acerca del significado de esa palabra?
o Meta: encontrar y usar el glosario en español para aprender más acerca de un término complicado.
¿Qué haría primero? ¿Por qué?
¿Qué está buscando?
¿En qué palabras está pensando?
¿Por qué fue a ese lugar específico del sitio Web? ¿Cuál sería el paso siguiente?
Nota para el facilitador: una vez que el participante haya llegado al glosario, pídale que brevemente
localice la definición de ITIN y luego hágale las preguntas que aparecen debajo. [Si el participante no
puede localizar el glosario, diríjalo a la página y pregúntele ¿qué retos encontró? ¿Por qué no pudo
localizar la página del glosario?
En una escala del 1 al 10, donde uno es muy difícil de entender y 10 muy fácil de entender,
¿cómo califica la información de esta página?
¿Cuáles son algunas de las razonas para darle un (diga el número)?
¿Cuán fácil o difícil fue encontrar el glosario?
¿Es el término “glosario” un término adecuado para usar cuando se hace referencia a un
diccionario de términos? ¿Por qué sí o por qué no?
¿Cuán útil fue la información? ¿Qué fue lo más útil? ¿Qué fue lo menos útil?
¿Qué le gustaría ver?
¿Alguna sugerencia para facilitar esta tarea?
Escenario 5: Cómo localizar la información de contacto para pedir ayuda (Entregue
el papel con el escenario 5). Por favor léalo en voz alta.
Le gustaría obtener ayuda personalizada acerca de la información del VIH/SIDA vía Internet o por
teléfono. ¿En dónde encontraría esta información?
o Meta: determinar si el participante entiende que hay disponibles servicios de “chat” en vivo vía
ayuda en vivo.
¿Qué haría primero? ¿Por qué?
B-20
¿Qué está buscando?
¿En qué palabras está pensando?
¿Por qué fue a ese lugar específico del sitio Web? ¿Cuál sería el paso siguiente?
Nota para el facilitador: en la página de ayuda en vivo, pídale que lea brevemente la información de la
página y hágale las preguntas que aparecen debajo. [Si el participante no puede localizar "ayuda en
vivo”, diríjalo a la página y pregúntele ¿qué retos encontró? ¿Por qué no pudo localizar la “ayuda en
vivo”?
o Meta: determinar si: 1) el término “ayuda en vivo” es adecuado, 2) el participante entiende que la
“ayuda en vivo” es un servicio de ‘chat’, 3) el lugar de la “ayuda en vivo” y el número 1- 800 son
donde las personas buscan ayuda y, 4) el participante sabe que hay otros métodos de ayuda
disponibles además de “ayuda en vivo” (por ejemplo el teléfono o el correo electrónico).
En una escala del 1 al 10, donde uno es muy difícil de entender y 10 muy fácil de entender,
¿cómo califica la información de esta página?
¿Cuáles son algunas de las razones para darle un (diga el número)?
Cuando ve la frase “ayuda en vivo”, ¿que se le viene a la mente?
¿Cuán fácil fue localizar el número 1-800?
En una escala del 1 al 10, en donde uno es, “no fue útil para nada” y diez, “muy útil”, ¿cómo
califica la información de esta página?
¿Cuáles son algunas de las razones para darle un (diga el número)?
¿Qué le gustaría ver?
¿Alguna sugerencia para facilitar esta tarea?
Escenario 6: Formatos para celulares
Una de las áreas que nos interesa es saber más acerca de cómo los proveedores usan sus servicios de
celular. ¿Usa un smart phone (Blackberry, iPhone), iPod Touch u otro aparato celular?
Si contestó sí:
• ¿Con qué frecuencia lo usa para acceder la información sobre un protocolo de
salud?
• Sondeo: ¿cuáles son los sitios de información de salud que accede
típicamente?
• ¿Qué le gusta o no le gusta de los sitios que accede en su aparato celular?
Sondeo: ¿qué los hace más fáciles o más difíciles de usar?
• Si fuéramos a lanzar una versión en infoSIDA para celulares, ¿qué partes del sitio le
gustaría ver en un aparato celular?
Sondeo: ¿sería beneficioso incluir en un aparato celular hojas informativas,
glosarios, ensayos clínicos, temas de salud, páginas especializadas y las
herramientas de traducción? ¿Por qué sí o por qué no?
¿Ha visto que otros sitios de información de salud provean este servicio vía
un teléfono celular? ¿Qué le gustó o no le gustó?
• ¿Qué tal la base de datos de medicamentos?
B-21
Sondeo: ¿que otros sitios con base de datos visita en su aparato celular?
¿Qué le gusta o no le gusta de cómo su celular se interconecta?
Cierre falso
Como mencioné anteriormente, hay algunos observadores que están mirando nuestro debate. Me voy a
salir del salón por un minuto para ver si ellos tienen alguna otra pregunta que les gustaría que le hiciera.
Finalización
¿Tiene algún otro comentario que quisiera compartir conmigo acerca de qué le gusta o no le gusta del
sitio Web de infoSIDA o cómo le gustaría que mejorara?
A nombre de la Biblioteca Nacional de Medicina, le agradezco su tiempo. Realmente apreciamos su
aporte para mejorar este sitio. Asegúrese de buscar a __________ a su salida para su remuneración.
Gracias de nuevo.
B-22
Appendix C: Informed Consent Form
INFORMED CONSENT – English*
Usability Testing of AIDSinfo and infoSIDA
November/December 2010
ICF International is conducting usability testing on behalf of the National Library of Medicine (NLM). You
have been invited to participate in a 90-minute session to explore and evaluate a Web site with health
information about HIV and AIDS. The Web site is available in both English and Spanish.
We have invited two sets of nine people to review each of the English and Spanish Web sites, so you are
one of nine members of the public who will be reviewing the site you will see today. AIDSinfo is being
reviewed by healthcare professionals such as physicians, nurse practitioners, and pharmacists. infoSIDA
is being reviewed by healthcare providers as well as patients, caregivers, educators and community
outreach professionals.
Your opinions and input will help us organize the structure of the site, the layout of the pages, and the
information presented to help people become comfortable using the site. We expect that as a result of
your participation, your feedback will be used to redesign the Web site so that the information will
become more relevant to users, answers to people’s questions will be more easily retrieved, and
commonly requested information will be easier to locate.
Before you agree to join in this discussion, please review and consider the conditions listed below:
• Participation in this study is completely voluntary. You are not required to answer any questions you
don’t want to answer or perform any tasks you don’t wish to perform. You are free to leave at any
time.
• To assist the project team in identifying people’s preferences and noting trends in how people use the
site, your on-screen interactions will be recorded to hard disk. This recording will be synchronized
with an audio recording of your conversation with the usability test administrator. A small Webcam
view of your face will also be recorded so the Web site designers can note your facial expressions like
surprise or interest.
• The recordings will only be viewed by the ICF project team preparing a written report on how to
improve the Web site and by the people at NLM responsible for developing and evaluating the Web
site. Your name and personal information will NOT be associated either with your recording or with
your reported experience with the site. In any written report or briefing to our client resulting from
this study, all findings regarding your experience with the site will be reported in conjunction with
that of eight other participants in the study.
• In the event ICF or our client have future questions about the study or what we observed today, a
DVD recording of your session will be archived at ICF, the company conducting this testing. This
recording will remain protected in a secure storage in case of the need for future review; they will not
be made available to the public. Only the members of the ICF and NLM research team will have
access to the recordings, and they will sign an agreement to protect them from release to others.
• Today’s test session may be observed live by project staff from the ICF and NLM over a video
connection.
• We ask you to avoid using your last name during the session.
C-1
• The risks to you from participating in this research are minimal, and there are no direct benefits, other
than that you will receive an incentive payment of [$100 to $350--varies by participant category] for
your time.
• Please be sure you ask any questions you have about the study or this form, and that you are satisfied
with the answers before signing your consent to participate.
• You have a right to receive a copy of your signed consent form before you leave today.
Contact information: The procedures involved in this usability test have been examined by an
institutional review board representative to ensure protection of your personal information. If you
have any questions about this issue, your participation, or the project in general that are not answered
today, please contact Christina Copty at ICF International, telephone number (301) 315-2828.
Your signature below means that you understand the conditions stated above and agree to participate
in this study.
Signature_______________________________ Witness ____________________________________
Date _______________________________________
C-2
