Dedicated object processor for mobile augmented reality - sailor assistance case study

This paper addresses the design of embedded systems for outdoor augmented reality (AR) applications integrated to see-through glasses. The set of tasks includes object positioning, graphic computation, as well as wireless communications, and we consider constraints such as real-time, low power, and low footprint. We introduce an original sailor assistance application, as a typical, useful, and complex outdoor AR application, where context-dependent virtual objects must be placed in the user field of view according to head motions and ambient information. Our study demonstrates that it is worth working on power optimization, since the embedded system based on a standard general-purpose processor (GPP) + graphics processing unit (GPU) consumes more than high-luminosity see-through glasses. This work presents then three main contributions, the first one is the choice and combinations of position and attitude algorithms that fit with the application context. The second one is the architecture of the embedded system, where it is introduced as a fast and simple object processor (OP) optimized for the domain of mobile AR. Finally, the OP implements a new pixel rendering method (incremental pixel shader (IPS)), which is implemented in hardware and takes full advantage of OpenGL ES light model. A GP+OP(s) complete architecture is described and prototyped on field programmable gate-array (FPGA). It includes hardware/software partitioning based on the analysis of application requirements and ergonomics.

[1]  Robert B. McGhee,et al.  An extended Kalman filter for quaternion-based orientation estimation using MARG sensors , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[2]  N. D. Durie,et al.  Digest of papers , 1976 .

[3]  Di Li,et al.  Low-cost MEMS sensor-based attitude determination system by integration of magnetometers and GPS: A real-data test and performance evaluation , 2008, 2008 IEEE/ION Position, Location and Navigation Symposium.

[4]  Willem H. Steyn,et al.  Kalman filter configurations for a low-cost loosely integrated inertial navigation system on an airship , 2008 .

[5]  Dave Shreiner,et al.  The OpenGL ES 2.0 programming guide , 2008 .

[6]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[7]  Yu-Te Liao,et al.  A single-pixel wireless contact lens display , 2011, Journal of Micromechanics and Microengineering.

[8]  Jack J. Purdum,et al.  C programming guide , 1983 .

[9]  Adam Herout,et al.  Fractal marker fields: No more scale limitations for fiduciary markers , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[10]  Sangkyung Sung,et al.  In-flight Heading Estimation of Strapdown Magnetometers using Particle Filters , 2008, 2008 IEEE National Aerospace and Electronics Conference.

[11]  Russell Tessier,et al.  A security approach for off-chip memory in embedded microprocessor systems , 2009, Microprocess. Microsystems.

[12]  D. Gebre-Egziabher,et al.  A gyro-free quaternion-based attitude determination system suitable for implementation using low cost sensors , 2000, IEEE 2000. Position Location and Navigation Symposium (Cat. No.00CH37062).

[13]  J. Gregory Steffan,et al.  The Potential for a GPU-Like Overlay Architecture for FPGAs , 2011, Int. J. Reconfigurable Comput..

[14]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[15]  K. Sarayeddine,et al.  Key challenges to affordable see-through wearable displays: the missing link for mobile AR mass deployment , 2013, Defense, Security, and Sensing.

[16]  Matthew Franklin,et al.  The Lessons Learned in the Application of Augmented Reality , 2006 .

[17]  Jan Skaloud,et al.  Assessment of the Integration Strategy between GPS and Body-Worn MEMS Sensors with Application to Sports , 2007 .

[18]  Shoji Ikeda,et al.  Fabrication of a magnetic tunnel junction-based 240-tile nonvolatile field-programmable gate array chip skipping wasted write operations for greedy power-reduced logic applications , 2013, IEICE Electron. Express.

[19]  Jang Gyu Lee,et al.  Adaptive Two-Stage Extended Kalman Filter for a Fault-Tolerant INS-GPS Loosely Coupled System , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[20]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[21]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[22]  Xin Yang,et al.  LDB: An ultra-fast feature for scalable Augmented Reality on mobile devices , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[23]  Eun-Hwan Shin,et al.  An unscented Kalman filter for in-motion alignment of low-cost IMUs , 2004, PLANS 2004. Position Location and Navigation Symposium (IEEE Cat. No.04CH37556).

[24]  R. Schroer,et al.  Position, Location, and Navigation Symposium (PLANS) , 2004 .

[25]  Hugo De Man,et al.  Formalized methodology for data reuse: exploration for low-power hierarchical memory mappings , 1998, IEEE Trans. Very Large Scale Integr. Syst..

[26]  Steven Zhiying Zhou,et al.  Positioning, tracking and mapping for outdoor augmentation , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[27]  Zhang Ren,et al.  The soft iron and hard iron calibration method using extended kalman filter for attitude and heading reference system , 2008, 2008 IEEE/ION Position, Location and Navigation Symposium.

[28]  S. Nasiri.  A Critical Review of MEMS Gyroscopes Technology and Commercialization Status , 2005 .