Intelligent Vision Systems – Bringing Human-Machine Interface to AR/VR

AR/VR devices are believed to be the next generation mobile compute platform that will replace today’s smart-phones. These devices will require new human-machine input (HMI) modalities that are intuitive, non-interruptive, and socially friendly and responsive in an all-day wearable glasses form factor. The candidate HMI modalities include voice, eye gazing, hand/head/body gestsures, and brain-computer interface (BCI). This paper describes computer vision based modalities along with the sensor and system specifications. It also proposes solutions to the extremely stringent power, form factor and performance challenges.

[2]  H. Kawashima,et al.  Pixel/DRAM/logic 3-layer stacked CMOS image sensor technology , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[3]  Xin Wang,et al.  5.1 A Stacked Global-Shutter CMOS Imager with SC-Type Hybrid-GS Pixel and Self-Knee Point Calibration Single Frame HDR and On-Chip Binarization Algorithm for Smart Vision Applications , 2019, 2019 IEEE International Solid- State Circuits Conference - (ISSCC).

[4]  Richard A. Newcombe,et al.  Sensors for Future VR Applications , 2017 .

[5]  R. Fontaine.  A Survey of Enabling Technologies in Successful Consumer Digital Imaging Products , .

[6]  Hongtao Han,et al.  Wafer Level Camera technology - from wafer level packaging to wafer level integration , 2010, 2010 11th International Conference on Electronic Packaging Technology & High Density Packaging.

[7]  Mathias Beike,et al.  Digital Integrated Circuits A Design Perspective , 2016 .

[8]  Sergey Velichko,et al.  Low Noise High Efficiency 3.75 µm and 2.8 µm Global Shutter CMOS Pixel Arrays , 2013 .

[9]  Tae-Chan Kim,et al.  Advanced image sensor technology for pixel scaling down toward 1.0µm (Invited) , 2008, 2008 IEEE International Electron Devices Meeting.

[10]  Joseph G. Makin,et al.  Real-time decoding of question-and-answer speech dialogue using human cortical activity , 2019, Nature Communications.

[11]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[12]  Eric R. Fossum,et al.  Active pixel sensors: are CCDs dinosaurs? , 1993, Electronic Imaging.

[13]  Yichen Wei,et al.  Simple Baselines for Human Pose Estimation and Tracking , 2018, ECCV.