Improving kNN for Human Activity Recognition with Privileged Learning Using Translation Models

Multiple sensor modalities provide more accurate Human Activity Recognition (HAR) compared to using a single modality, yet the latter is preferred by consumers as it is more convenient and less intrusive. This presents a challenge to researchers, as a single modality is likely to pick up movement that is both relevant as well as extraneous to the human activity being tracked and lead to poorer performance. The goal of an optimal HAR solution is therefore to utilise the fewest sensors at deployment, while maintaining performance levels achievable using all available sensors. To this end, we introduce two translation approaches, capable of generating missing modalities from available modalities. These can be used to generate missing or “privileged” modalities at deployment to augment case representations and improve HAR. We evaluate the presented translators with k-NN classifiers on two HAR datasets and achieve up-to \(5\%\) performance improvements using representations augmented with privileged modalities. This suggests that non-intrusive modalities suited for deployment benefit from translation models that generates privileged modalities.