The Usefulness of User Testing Methods in Identifying Problems on University Websites

Layla Hasan

Abstract


This paper aims to investigate the usefulness of three user testing methods (observation, and using both quantitative and qualitative data from a post-test questionnaire) in terms of their ability or inability to find specific usability problems on university websites. The results showed that observation was the best method, compared to the other two, in identifying large numbers of major and minor usability problems on university websites. The results also showed that employing qualitative data from a post-test questionnaire was a useful complementary method since this identified additional usability problems that were not identified by the observation method. However, the results showed that the quantitative data from the post-test questionnaire were inaccurate and ineffective in terms of identifying usability problems on such websites.

Keywords


Usefulness; user testing; methods; university websites; usability; comparison

Full Text:

PDF

References


Alexander, D. (2005). How Usable are University Websites? A Report on a Study of the Prospective Student Experience. Technical Report. Monash University.

Alva, M., Martínez, A., Cueva, J.M., Sagástegui, C., and López, B. (2003). Comparison of Methods and Existing Tools for the Measurement of Usability in the Web. In the Proceedings of the 3rd International Conference on Web Engineering (ICWE’03), Spain, Springer, 386-389.

Bidgoli, H. (2004). The Internet Encyclopaedia. John Wiley and Sons.

Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale. In: P.W. Jordan, B. Thomas, B.A. Weerdmeester & I.L. McClelland (Eds.), Usability Evaluation in Industry. London: Taylor and Francis.

Chaparro, B. (2008). Usability Evaluation of a University Portal Website. Usability News, 10(2), 1-7.

Chin, J.P., Diehl, V.A. and Norman, K.L. (1988). Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface. In: Soloway E, Frye D, Sheppard SB, editors. In the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Washington, DC. New York, ACM Press, 213-221.

Christoun, S., Aubin, H., Hannon, C., and Wolk, R. (2006). Web Site Usability in Higher Education. Information Systems Education Journal, 4(110).

Doll, W.J. and Torkzadeh. G. (1988). The Measurement of End-user Computing Satisfaction. MIS Quarterly, 12(2), 259-274.

Dumas, J. S. and Redish, J. C. (1999). A Practical Guide to Usability Testing, Second Intellect Ltd, Rev Sub Edition.

Finstad, K., (2006). The System Usability Scale and Non-Native English Speakers. Journal of Usability Studies, 1(4), 185-188.

Goodwin, S. (2005). Using Screen Capture Software for Website Usability and Redesign Buy-in. Library Hi Tech, 23(4), 610-621.

Gray, W. and Salzman, C. (1998). Damaged Merchandise? A Review of Experiments that Compare Usability Evaluation Methods. Human-Computer Interaction, 13, 203-261.

Eduroute. (2011). http://www.eduroute.info. [accessed 14.02.2011].

Harper, B., Slaughter, L., and Norman, K. (1997). Questionnaire Administration via the WWW: A Validation & Reliability Study for a User Satisfaction Questionnaire. In: Lobodzinski S, Tomek I, editors. In the Proceedings of the World Conference on the WWW, Internet & Intranet, Toronto, CA. Charlottesville (VA), AACE, 808-818.

Hasan, L. (2014). Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics. The International Arab Journal of e-Technology (IAJeT), 3(3), 179-193.

Hasan, L. (2013). Heuristic Evaluation of Three Jordanian University Websites. Informatics in Education, 12(2), 231–251.

Hasan, L., Morris, A., and Probets, S. (2012). A Comparison of Usability Evaluation Methods for Evaluating E-Commerce Websites. Behaviour & Information Technology Journal, 31(7), 707–737.

Holzinger, A. (2005). Usability Engineering Methods for Software Developers. Communications of the ACM, 48(1), 92-99.

ISO 9241-11. (1998). International Standard. First Edition, Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs). Part11: Guidance on Usability. http://www.idemployee.id.tue.nl/g.w.m. rauterberg/lecturenotes/ISO9241part11.pdf. [accessed 03.04.2010].

Kasli, M. and Avcikurt, C. (2008). An Investigation to Evaluate the Websites of Tourism Departments of Universities in Turkey. Journal of Hospitality, Leisure, Sport and Tourism Education, 7(2), 77-92.

Kirakowski, J. and Corbett, M. (1993). SUMI: The Software Measurement Inventory. British Journal of Education Technology, (24), 210-212.

Kirakowski, J. (1995). The Software Usability Measurement Inventory: Background and Usage, in Usability Evaluation in Industry, Jordan, P., Thomas, B. and Weerdmeester, B. (eds.), Taylor & Frances, London.

Lazar, J. (2006). Web Usability: A User-Centered Design Approach. Pearson/Education Inc.

Lencastre J. and Chaves J. (2008). A Usability Evaluation of Educational Websites. in the Proceedings of EADTU Conference. France.

Lewis, J.R. (1995). IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human Computer Interaction, 7(1), 57-78.

Mustafa, S. and Al-Zoua'bi, L. (2008). Usability of the Academic Websites of Jordan's Universities. In the Proceedings of the International Arab Conference on Information Technology. Tunisia.

Nielsen, J. (1993). Usability Engineering. London: Academic Press.

Nielsen, J. (2003). Usability 101: Introduction to Usability. Useit.com, http://www.useit.com/alertbox/20030825 .html. [accessed 14.02.2011].

Nielsen, J. and Mack, R. L. (1994). (Eds.). Usability Inspection Methods, John Wiley & Sons. New York.

Nielsen, J. and Pernice, K. (2010). Eyetracking Web Usability. New Riders Press. ISBN 0-321-49836-4.

Rubin, J. (2008). Handbook of Usability Testing: How to Plan. Design, and Conduct Effective Tests. Wiley Publishing Inc.

Rogers, Y., Sharp, H., and Preece, J. (2011). Interaction Design: Beyond Human-Computer Interaction. Wiley. Third Edition.

Sauro, J., (2011). Measuring Usability with the System Usability Scale (SUS). https://www.measuringusability.com/sus.php#comments. [accessed 24.07.2014].

Stone, D., Jarrett, C., Woodroffe, M., and Minocha S. (2005). User Interface Design and Evaluation. The Open University. Morgan Kaufmann.

TechSmith. (2014). http://www.techsmith.com/morae.html. [accessed 22.07.2014].

Tullis, T. S. and Stetson, J. N. (2004). A Comparison of Questionnaires for Assessing Website Usability, In the Proceedings of the Usability Professionals Association (UPA) 2004 Conference, Minneapolis, USA.

Tüzün, H., Akinci, A., Kurtoğlu, M., Atal, D., and Pala, F. (2013). A Study on the Usability of a University Registrar’s Office Website Through the Methods of Authentic Tasks and Eyetracking. The Turkish Online Journal of Educational Technology (TOJET), 12(2), 26-38.

Van den Haak, M. and de Jong, M. (2005). Analyzing the Interaction between Facilitator and Participants in Two Variants of the Think-Aloud Method. In the Proceedings of IEEE International Professional Communication Conference, 323-327.

Veenendall Van E. (1998). Questionnaire based Usability Testing. In: Unknown, editor. EURO & Y2K: The Industrial Impact. In the Proceedings of the the European Software Quality Week, Brussels, BE. San Francisco (CA): Software Research, Inc. 1-9.

WAMMI. (2014). http://www.wammi.com. [accessed 22.07.2014].




DOI: http://dx.doi.org/10.4301/s1807-17752014000200002

Copyright (c)

Licensed under