PrISM-Tracker

Citations Over TimeTop 12% of 2022 papers

Abstract

A user often needs training and guidance while performing several daily life procedures, e.g., cooking, setting up a new appliance, or doing a COVID test. Watch-based human activity recognition (HAR) can track users' actions during these procedures. However, out of the box, state-of-the-art HAR struggles from noisy data and less-expressive actions that are often part of daily life tasks. This paper proposes PrISM-Tracker, a procedure-tracking framework that augments existing HAR models with (1) graph-based procedure representation and (2) a user-interaction module to handle model uncertainty. Specifically, PrISM-Tracker extends a Viterbi algorithm to update state probabilities based on time-series HAR outputs by leveraging the graph representation that embeds time information as prior. Moreover, the model identifies moments or classes of uncertainty and asks the user for guidance to improve tracking accuracy. We tested PrISM-Tracker in two procedures: latte-making in an engineering lab study and wound care for skin cancer patients at a clinic. The results showed the effectiveness of the proposed algorithm utilizing transition graphs in tracking steps and the efficacy of using simulated human input to enhance performance. This work is the first step toward human-in-the-loop intelligent systems for guiding users while performing new and complicated procedural tasks.

Related Papers

• → The Application of Baum-Welch Algorithm in Multistep Attack(2014)24 cited
• Gesture Classification Using Hidden Markov Models and Viterbi Path Counting(2003)
• → A Viterbi algorithm for a trajectory model derived from HMM with explicit relationship between static and dynamic features(2004)27 cited
• Particle filters for HMM state inference(2012)
• → Information Extraction from Chinese Papers Based on Hidden Markov Model(2013)2 cited