To main content

Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series

Abstract

The field of time series anomaly detection is constantly advancing, with several methods available, making it a challenge to determine the most appropriate method for a specific domain. The evaluation of these methods is facilitated by the use of metrics, which vary widely in their properties. Despite the existence of new evaluation metrics, there is limited agreement on which metrics are best suited for specific scenarios and domains, and the most commonly used metrics have faced criticism in the literature. This paper provides a comprehensive overview of the metrics used for the evaluation of time series anomaly detection methods, and also defines a taxonomy of these based on how they are calculated. By defining a set of properties for evaluation metrics and a set of specific case studies and experiments, twenty metrics are analyzed and discussed in detail, highlighting the unique suitability of each for specific tasks. Through extensive experimentation and analysis, this paper argues that the choice of evaluation metric must be made with care, taking into account the specific requirements of the task at hand.
Read the publication

Category

Academic article

Language

English

Affiliation

  • SINTEF Digital / Software Engineering, Safety and Security
  • Norwegian University of Science and Technology

Year

2023

Published in

Data mining and knowledge discovery

ISSN

1384-5810

Volume

38

View this publication at Norwegian Research Information Repository