Visualization system for sound field using see-through head-mounted display

For the visualization of a sound field, a widely used method is the superimposition of the sound information onto a camera view. Although it effectively enables the understanding the relationship between space and sound, a planar display cannot resolve depth information in a straightforward manner. In contrast, a see-through head-mounted display (STHMD) is capable of representing three-dimensional (3D) vision and natural augmented reality (AR) or mixed reality (MR). In this paper, we propose a system for the measurement and visualization of a sound field with an STHMD. We created two visualization systems using different types of STHMDs and technologies for realizing AR/MR and a measurement system for a 3D sound intensity map, which can be used together with the visualization system. Through three visualization experiments, we empirically found that the stereoscopic viewing and the convenient viewpoint movement associated with the STHMD enables understanding of the sound field in a short time.

[1]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[2]  Yasuhiro Oikawa,et al.  Visualization of 3D sound field using see-through head mounted display , 2017, SIGGRAPH Posters.

[3]  S H Park,et al.  Visualization of pass-by noise by means of moving frame acoustic holography. , 2001, The Journal of the Acoustical Society of America.

[4]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[5]  Keith Holland,et al.  Scan and Paint: Theory and Practice of a Sound Field Visualization Method , 2013 .

[6]  Kung Yao,et al.  Source localization and beamforming , 2002, IEEE Signal Process. Mag..

[7]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[8]  Rae A. Earnshaw,et al.  Virtual Reality Systems , 1993 .

[9]  David W. Murray,et al.  Video-rate localization in multiple maps for wearable augmented reality , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[10]  J. Chung,et al.  Cross‐spectral method of measuring acoustic intensity without error caused by instrument phase mismatch , 1978 .

[11]  H. R. Harrison,et al.  Quaternions and Rotation Sequences: a Primer with Applications to Orbits, Aerospace and Virtual Reality , J. B. Kuipers, Princeton University Press, 41 William Street, Princeton, NJ 08540, USA. 1999. 372pp. Illustrated. £35.00. ISBN 0-691-05872-5. , 1999, The Aeronautical Journal (1968).

[12]  Yasuhiro Oikawa,et al.  Sound field measurements based on reconstruction from laser projections , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[13]  Patricia Davies,et al.  An Efficient Procedure for Visualizing the Sound Field Radiated by Vehicles during Standardized Passby Tests , 2000 .

[14]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[15]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Bahram Javidi,et al.  A 3D integral imaging optical see-through head-mounted display. , 2014, Optics express.

[17]  J. D. Maynard,et al.  Nearfield acoustic holography: I. Theory of generalized holography and the development of NAH , 1985 .

[18]  Yasuhiro Oikawa,et al.  Optically visualized sound field reconstruction using Kirchhoff–Helmholtz equation , 2015 .

[19]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[20]  Yasuhiro Oikawa,et al.  Optical sound field measurement and imaging using laser and high-speed camera , 2016 .

[21]  K. Ishikawa,et al.  High-speed imaging of sound using parallel phase-shifting interferometry. , 2016, Optics express.

[22]  Finn Jacobsen,et al.  A comparison of two different sound intensity measurement principles , 2005 .

[23]  Naokazu Yokoya,et al.  A stereoscopic video see-through augmented reality system based on real-time vision-based registration , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[24]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[25]  Yasuhiro Oikawa,et al.  Three-dimensional sound-field visualization system using head mounted display and stereo camera , 2016, Proc. Meet. Acoust..

[26]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[27]  Yasuhiro Oikawa,et al.  Spatio-temporal filter bank for visualizing audible sound field by Schlieren method , 2017 .