To main content

Work-domain experts as evaluators: Usability inspection of domain-specific work-support systems

Abstract

Can work-domain experts generate high-impact evaluation results when used as evaluators in usability inspections of domain-specific work-support systems? This study investigated this question empirically. Three applications were evaluated with group-based expert walkthroughs. The walkthroughs were conducted under 2 conditions-with either work-domain experts or usability experts as evaluators. The condition with usability experts served as the background on which to evaluate the performance and impact of the condition with work-domain experts. The work-domain experts identified a smaller number of items (user problem and/or design suggestion) than the usability experts. However, the items identified by the work-domain experts were on average classified as more severe, and the developers (groups or organizations carrying out subsequent development) gave higher priority to items identified by work-domain experts. As a consequence of the higher severity classification and priority given to the work-domain experts' items, it was concluded that work-domain experts may indeed produce high-impact evaluation results when used as evaluators in a usability inspection. The conclusion opens up exciting method development possibilities in the area of usability inspection methods. The study's research design also represents a fresh research approach to the evaluation of usability evaluation methods, utilizing the impact of the evaluation results in the subsequent development process as an evaluation criterion

Category

Academic article

Language

English

Author(s)

Affiliation

  • SINTEF Digital / Software and Service Innovation

Year

2007

Published in

International Journal of Human-Computer Interaction

ISSN

1044-7318

Volume

22

Issue

3

Page(s)

217 - 245

View this publication at Cristin