To main content

Monitoring In-Home Emergency Situation and Preserve Privacy using Multi-modal Sensing and Deep Learning

Abstract

Videos and images are commonly used in home monitoring systems. However, detecting emergencies in-home while preserving privacy is a challenging task concerning Human Activity Recognition (HAR). In recent years, HAR combined with deep learning has drawn much attention from the general public. Besides that, relying entirely on a single sensor modal-ity is not promising. In this paper, depth images and radar presence data were used to investigate if such sensor data can tackle the challenge of a system's ability to detect abnormal and normal situations while preserving privacy. The recurrence plots and wavelet transformations were used to make a two-dimensional representation of the presence radar data. Moreover, we fused data from both sensors using data-level, feature-level, and decision-level fusions. The decision-level fusion showed its superiority over the other two techniques. For the decision-level fusion, a combination of the depth images and presence data recurrence plots trained first on convolutional neural networks (CNN). The output was fed into support vector machines, which yielded the best accuracy of 99.98%.
Read the publication

Category

Academic chapter

Language

English

Author(s)

  • David Andreas Bordvik
  • Jie Hou
  • Farzan Majeed Noori
  • Md Zia Uddin
  • Jim Tørresen

Affiliation

  • SINTEF Digital / Sustainable Communication Technologies
  • University of Oslo
  • Oslo University Hospital

Year

2022

Publisher

IEEE (Institute of Electrical and Electronics Engineers)

Book

2022 International Conference on Electronics, Information, and Communication (ICEIC 2022)

ISBN

9781665409353

View this publication at Norwegian Research Information Repository