Usability Evaluation Resources

Starting a usability initiative? There is great research on how to set up a useful and appropriate evaluation program. There are also websites with sample testing templates. Below is a list of resources I put together for a company beginning such a program. I think the resources listed below are helpful to anyone wanting to measure and improve the usability of their website, application, or online service.

Here are the resources in order of how to set up a usability program.

1) Prove the value of usability

Making the business case for usability:
* Bias, R. G., & Mayhew, D. J. (2005). Cost-Justifying Usability: An Update for the Internet Age, (2nd ed.). Morgan Kaufmann.

A sample report demonstrating a usability competitive analysis using individual and composite usability scores: Ecommerce Usability for High Street Retailers 2009

A study showing that lab usability testing can indeed uncover usability problems in the field.
* Marshall, C., McManus, B., & Prail, A. (1990). Usability of product X-lessons from a real product. Behaviour & Information Technology, 9(3), 243.

2) Measure the right things in the right way

Determine the appropriate usability metrics and how to best design a test to get at them. This book also has a few effective case studies on the value of usability testing:
* Tullis, T., & Albert, B. (2008). Measuring the user experience: Collecting, analyzing, and presenting usability metrics. Boston: Elsevier.

More info on how to determine appropriate metrics and the challenges of measuring them correctly:
* Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79-102.
* Hughes, M. (1999). Rigor in usability testing. Technical Communication, 46 (4), 488–494.

Information on a common industry measure, the SUS score:
* Determining What Individual System Usability Scale Scores Mean: Adding an Adjective Rating Scale Journal of Usability Studies, Volume 4, Issue 3, May 2009, pp. 114-123

For a discussion on the problems of relying entirely on user self-reporting read usability guru Jakob Nielsen's First Rule of Usability? Don't Listen to Users

3) Set up your usability evaluation reporting

Templates for website usability evaluation.

Tailored for retail, but this sample demonstrates how a template can
be customized for a particular genre or website model.

User testing protocol with and applicable template.

Sample test script

4) Additional considerations

A useful article on the value of merely discovering usability problems or posing redesign consideration to developers. Also considers expert review vs. user testing:
* Hornbæk, K., & Frøkjær, E. (2005). Comparing usability problems and redesign proposals as input to practical systems development. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 391-400). Portland, Oregon, USA: ACM.

Can outside usability experts identify the intricacies of one's business domain or are internal specialists required?
* Følstad, A., Anda, B. C., & Sjøberg, D. I. (2010). The usability inspection performance of work-domain experts: An empirical study. Interacting with Computers, 22(2), 75-87.

Current limitations with usability studies and what is needed in the future:
* Lund, A. M. (2006). Post-modern usability. Journal of Usability Studies, 2(1), 1–6.

For insight on usability metrics and innovative measure (ie. social, hedonic, affective) see the chapters by Cockton and Lindgaard in:
* Law, E. L., Hvannberg, E. T., Cockton, G., Jeffries, R., & Wixon, D. (2007). Maturing Usability: Quality in Software, Interaction and Value. London: Springer.