You've developed your Usability Test, scheduled your test subjects
and now your are ready to go, right? But before you bring in your
first test subject, you have three important quality checks.
1) Test the Usability Test - sit down at your computer
with the usability test in hand, read the questions to yourself, and
attempt to do all the tasks. Does the test work for you? Or are parts
of the web site not ready for prime time yet? Share the Usability
Test with your web site developers so they know what sections of the
site you are testing. Make sure that the site will be stable and ready
during the dates you are conducting the usability test.
The site doesn't need to be perfect, or finished. Just make sure
your questions/tasks are actually doable, and not just a dead end.
2) Practice Giving the Test - grab an unsuspecting
co-worker or friend and conduct the entire usability test on them.
This will help you feel more comfortable when you do the test on your
first real test subject. Have your friend/co-worker complete all the
paper work (waiver, entrance and exit questions). As you read the
task based questions aloud, remember not to lead them to the answer.
Encourage them to talk out loud (with some folks you will need to
encourage them to do this on every question).
Remind them over and over, that you are not testing them, but you
are testing the software. If they have problems completing the tasks,
their complaints and frustrations (voiced aloud) will help you convince
the developers that things need to be changed.
Tell them you want them to be brutally honest. You had nothing to
do with the design of the system and want them to tell you exactly
what they are thinking. If they think the site "stinks"
or they are ready to pull their hair out, you want them to tell you!
As they give you feedback, both negative and positive, tell them
"that is a good point" and write down what they say. Show
approval and appreciation for their comments.
3) Supply Checklist - make sure you
have everything you need to conduct each test. Your checklist might
look something like this:
I like to create a web page that has copies of all the usability testing
documents online as well as a link to the site. Then, if I have forgotten
anything, I can grab a copy online and print it out wherever I am.
Okay, now we are ready to conduct the test.
1) Welcome: Welcome the test subject and thank them
for coming. Make them feel at ease. Tell a joke, or talk about the weather.
Ask if they have ever been in a usability study before. Assure them
that it is fun and easy.
2) Agenda: Outline the main things you will be doing
during the usability study. For example, you could say:
First, I will have you sign a waiver that indicates your willingness
to participate in this usability study and let's you know that I'll
be taking notes of your comments, but will keep all your personal
information private.
Second, I will ask you some basic demographic questions.
Third, I will ask you to complete X number of tasks on the new web
site. Keep in mind that we aren't testing you, but we are testing
the web site. Any problems or frustrations you encounter will help
us see where the design needs to be changed.
Fourth, I will ask you for your general feedback on the site. We
want to know your opinion.
This process should take about an hour, so let's get started. (Note:
It is important to end the test on time. You need to be respectful of
the persons time, especially if you want to get honest answers.)
3) Waiver/Entrance Questions: Have the person complete
the waiver. Answer any questions they have. Have the person complete
the demographic/entrance questions. You can either have them complete
the form on paper, or ask them the questions out loud and fill it in
for them. Whichever works best for you.
4) Task Based Questions: Have a computer ready with
the browser open and sitting on the starting page of your web site.
As the test facilitator, you will instruct and observe the subjects
performing fairly simple, common tasks. You will verbally lead them
through the series of tasks/questions, encouraging them to think out
loud and respond to what they are looking at. You will ask questions
about their thought processes and their decisions as they work, without
being intrusive or leading. You will also take in-depth notes directly
on the Task Based Questionnaire.
Things "To Do" and Things "Not to Do" while facilitating
the test:
| Things to Do |
Things Not To Do |
| Listen carefully |
Fail to Listen |
| Encourage |
Criticize |
| Be Neutral |
Be Defensive |
| Speak English |
Speak Geek |
| Answer questions with "What would you do?" or "What
do you think?" |
Lead user to the answer |
| Be Patient |
Be Impatience, Rush |
As you ask each question aloud, try to use the exact words on the test.
Do not lead the user to the answer. Do not help the user answer the
question. Remind the user we are testing that software, not them. Encourage
them to think out loud. (“what words are going thru your mind?”,
“what are you looking for?”)
Make note of the click stream (the path the users follows to complete
the task). Note any of the users comments and suggestions for making
the task easier. Finally, indicate if the person was able to successful
complete the task, as well as your opinion of their satisfaction or
frustration level.
Remember, someone will have to transcribe these notes. So try to write
legible and capture what you are observing.
5) Exit Questions:
When users are finished going through the set of task-based questions,
hand them the exit questions and ask them to answer them. Then give
them a few minutes to talk informally about their usability experience
with you. Ask them what they liked/disliked about the site most, if
they have suggestions for improvements, etc. If they’ve done or
mentioned anything during the test itself that you want to ask further
questions about, now is the time to ask. In my experience, users are
only too happy to talk with you when the test is over!
