Exploring a Mixed Reality Framework for the Internet-of-Things: Toward Visualization and Interaction with Hybrid Objects and Avatars

Smart hyper-connected environments are becoming a central part of daily life in modern society. Such environments apply the internet-of-things (IoT) paradigm [1], which refers to the growing field of interconnected devices and the networking that supports smart, embedded applications. The IoT has many human-computer interaction (HCI) challenges [2], however, and central to these challenges is the need to provide more human-friendly approaches to communicating sensor information and meaningful visualizations of contextual states to users of IoT systems. Highly expressive, and engaging smart environment interfaces are uncommon, and this work applies mixed reality as a tool to better visualize and express the underlying behaviors and states within IoT smart devices. This extends the authors’ previous research [3], providing a new head-mounted display framework and interconnection architecture for an augmented reality representation of a physical IoT device, an IoT Avatar. The video submission demonstrates contributions for: i) an exploration of how mixed reality can be used to enrich smart spaces and hybrid objects, and ii) an early use case and functionality evaluation of a simple avatar hybrid smart object that expresses emotion through immersive media. It is expected that this research will help foster immersive and engaging human-centered interaction in future smart environments.