A Multi-Kinect System for Serious Game Development Using ROS and Unity

Game-based rehabilitation systems are often proposed as an appropriate technology for improving patients’ motivation during the therapy. They have potential to increase the intensity and frequency of physical activity of stroke patients. In this work we present a Unity integrated ROS-based multi-Kinect system for serious game development. A multi-camera setup offers the advantages of enabling the player (patient) to walk along a wider area and minimizes the risk of occlusion due to people moving along the camera’s detection area or even to the patient’s own body. The proposed system merges the user joint position provided by multiple Kinect V2 sensors and delivers the results to Unity. A linking mechanism was used so that Unity can listen to and inject messages into ROS environment. We present data from a preliminary technical validation, which consists of an interface controlled by hand movements. The results show that 98.8% of the data obtained by the system were consistent with the user’s actions.