Unleashing the Power of Shared Label Structures for Human Activity Recognition