Planning the assessment: Specific decisions
When creating your usability assessment plan, pay specific attention to where and when you will test, and what equipment you will use.
Where to test?
- A usability laboratory: Test participants, who match user profiles, perform a sequence of typical tasks recorded under control conditions (one-way mirror, video camera, etc.). Major limitations to this approach are the length of time it takes for recruiting participants and potential bias from using an artificial environment.
- A remote testing environment: Test through a remote medium such as the internet or an online meeting service (e.g., WebEx). Due to the intuitive limitations of this approach (e.g., limited visual feedback, degree of separation, software problems, environmental distractions, difficulty building rapport and trust), this method is generally used only when in-person testing is impossible (e.g., geographic location) or when you want to broaden your results to other geographical areas.
- “Real” environment: Test in an environment similar to what users actually experience. This condition may be the best approach since it will provide feedback for testing that is conducted in a similar context in which users will use the final product.
When to test?
- Summative evaluation – after a product is fully developed
- Formative evaluation – during the development process or product use
- When to test will depend on your assessment purpose.
| Purpose | Formative | Summative |
|---|---|---|
| Technology development | X |
|
| Technology purchase |
|
X |
| Choosing between two or more technologies |
|
X |
| User satisfaction with technology | X |
|
What equipment to use?
- With or without video camera
- With or without screen recording software (such as Camtasia)
- Voice recorder for think alouds, or other qualitative methods (e.g., focus groups).
