To main content

Tabular Data Anomaly Patterns

Abstract

One essential and challenging task in data science is data cleaning - the process of identifying and eliminating data anomalies. Different data types, data domains, data acquisition methods, and final purposes of data cleaning have resulted in different approaches in defining data anomalies in the literature. This paper proposes and describes a set of basic data anomalies in the form of anomaly patterns commonly encountered in tabular data, independently of the data domain, data acquisition technique, or the purpose of data cleaning. This set of anomalies can serve as a valuable basis for developing and enhancing software products that provide general-purpose data cleaning facilities and can provide a basis for comparing different tools aimed to support tabular data cleaning capabilities. Furthermore, this paper introduces a set of corresponding data operations suitable for addressing the identified anomaly patterns and introduces Grafterizer - a software framework that implements those data operations
Read publication

Category

Academic chapter/article/Conference paper

Client

  • EC/H2020 / 732590
  • EC/H2020 / 732003
  • EC/H2020 / 644497

Language

English

Author(s)

Affiliation

  • SINTEF Digital / Sustainable Communication Technologies

Year

2017

Publisher

IEEE (Institute of Electrical and Electronics Engineers)

Book

2017 International Conference on Big Data Innovations and Applications (Innovate-Data), Prague, Czech Republic, Czech Republic, 21-23 Aug. 2017

ISBN

978-1-5386-0960-6

Page(s)

25 - 34

View this publication at Cristin