A multi-sensor integrated head-mounted display setup for augmented reality applications

We present an HMD based AR system comprising visual and inertial sensors. The visual sensor is a camera pair and the inertial sensors consist of an accelerometer and a gyroscope. We discuss the temporal and spatial calibration issues that relate to such a system. We introduce simple yet effective methods for estimating the time lag between the camera and the inertial sensors and for estimating the relative pose between the camera and the inertial sensors. These methods do not require a complicated setup for data collection and involve simple equations to solve. Sample results are presented to demonstrate the visual performance of the system.

[1]  A. Tanju Erdem,et al.  Fusing Inertial Sensor Data in an Extended Kalman Filter for 3D Camera Tracking , 2015, IEEE Transactions on Image Processing.

[2]  Tom Drummond,et al.  Tightly integrated sensor fusion for robust visual tracking , 2004, Image Vis. Comput..

[3]  Stergios I. Roumeliotis,et al.  A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation , 2008, IEEE Transactions on Robotics.

[4]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[5]  Tsuneo Yoshikawa,et al.  Accurate image overlay on video see-through HMDs using vision and accelerometers , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[6]  Li Bai,et al.  A Sensor Fusion Framework Using Multiple Particle Filters for Video-Based Navigation , 2010, IEEE Transactions on Intelligent Transportation Systems.

[7]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[8]  Jorge Dias,et al.  Relative Pose Calibration Between Visual and Inertial Sensors , 2007, Int. J. Robotics Res..

[9]  Jeroen D. Hol,et al.  Pose Estimation and Calibration Algorithms for Vision and Inertial Sensors , 2008 .

[10]  A. Tanju Erdem,et al.  Internal calibration of a camera and an inertial measurement unit , 2013, 2013 21st Signal Processing and Communications Applications Conference (SIU).

[11]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[13]  Denis Pomorski,et al.  GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects , 2006, Inf. Fusion.