To main content

FLAMINKO: Usability evaluation of work-domain specific applications. Experiences and lessons learnt.

Abstract

Work-domain specific software solutions represent important challenges for usability evaluation methods; particularly related to the lack of context knowledge resources available to the usability experts involved in the evaluation. In order to meet this challenge we have investigated two methods where work-domain experts (end users with thorough experience of the context of use) are engaged as analysts in usability evaluations.• Group-based expert evaluation is an analytical evaluation method designed to allow non-usability experts to participate as evaluators.• Cooperative usability testing is a combined empirical and analytical method, designed to allow the participating user to reflect systematically on their experience with the test and the tested solution. The two evaluation methods were investigated on basis of their application in two cases: (1) The redesign of a PDA application for mobile sales personnel and (2) the development of an in-vehicle solution for emergency response units. The methods’ performance was investigated with regard to thoroughness, validity and impact – three well defined measures in the literature. The results from the two cases were consistent. Group-based expert walkthrough may be recommendable as a low-cost method for usability evaluation of work-domain specific solutions. However, the method missed important usability issues identified through empirical usability testing. Cooperative usability testing seemed in these cases to be a preferable alternative to regular usability testing, as the inclusion of user reflection served to increase the usability issues covered as well as positively affect the impact on the development. The study also gave results of relevance also outside the scope of investigating the methods in question. In particular they serve to raise concern regarding the adequacy of the validity measurement as applied within the field of Human–computer interaction.

Oppdragsgiver: Norwegian research project, the VERDIKT program

Category

Report

Client

  • SINTEF AS / 90B235.50

Language

English

Author(s)

Affiliation

  • SINTEF Digital / Software and Service Innovation

Year

2009

Publisher

SINTEF

Issue

A11624

ISBN

9788214044393

View this publication at Cristin