Stereo Event-Based Particle Tracking Velocimetry for 3D Fluid Flow Reconstruction

Existing Particle Imaging Velocimetry techniques require the use of high-speed cameras to reconstruct time-resolved fluid flows. These cameras provide high-resolution images at high frame rates, which generates bandwidth and memory issues. By capturing only changes in the brightness with a very low latency and at low data rate, event-based cameras have the ability to tackle such issues. In this paper, we present a new framework that retrieves dense 3D measurements of the fluid velocity field using a pair of event-based cameras. First, we track particles inside the two event sequences in order to estimate their 2D velocity in the two sequences of images. A stereo-matching step is then performed to retrieve their 3D positions. These intermediate outputs are incorporated into an optimization framework that also includes physically plausible regularizers, in order to retrieve the 3D velocity field. Extensive experiments on both simulated and real data demonstrate the efficacy of our approach.

[1]  Sang Youl Yoon,et al.  3D particle position and 3D velocity field measurement in a microvolume via the defocusing concept , 2006 .

[2]  Yi Li,et al.  A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence , 2008, 0804.1703.

[3]  Davide Scaramuzza,et al.  ESIM: an Open Event Camera Simulator , 2018, CoRL.

[4]  Francisco Pereira,et al.  Defocusing digital particle image velocimetry: a 3-component 3-dimensional DPIV measurement technique. Application to bubbly flows , 2000 .

[5]  Wolfgang Heidrich,et al.  Stochastic tomography and its applications in 3D imaging of mixing fluids , 2012, ACM Trans. Graph..

[6]  A. Prasad Particle image velocimetry , 2000 .

[7]  Fulvio Scarano Particle Image Velocimetry , 2010 .

[8]  N Machicoane,et al.  A simplified and versatile calibration method for multi-camera optical systems in 3D particle imaging. , 2019, The Review of scientific instruments.

[9]  Christoph Schnörr,et al.  A variational approach for particle tracking velocimetry , 2005 .

[10]  Kostas Daniilidis,et al.  Event-Based Visual Inertial Odometry , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Shree K. Nayar,et al.  Compressive Structured Light for Recovering Inhomogeneous Participating Media , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Nils Thuerey,et al.  ScalarFlow , 2019, ACM Trans. Graph..

[14]  C. Brücker 3D scanning PIV applied to an air flow in a motored engine using digital high-speed video , 1997 .

[15]  Konrad Schindler,et al.  Volumetric Flow Estimation for Incompressible Fluids Using the Stationary Stokes Equations , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Christoph Schnörr,et al.  On-Line Variational Estimation of Dynamical Fluid Flows with Physics-Based Spatio-temporal Regularization , 2006, DAGM-Symposium.

[17]  Leonardo P. Chamorro,et al.  On the transient dynamics of the wake and trajectory of free falling cones with various apex angles , 2015 .

[18]  Markus Raffel,et al.  Background-oriented schlieren (BOS) techniques , 2015 .

[19]  Markus Raffel,et al.  Principle and applications of the background oriented schlieren (BOS) method , 2001 .

[20]  Wolfgang Heidrich,et al.  TomoFluid: Reconstructing Dynamic Fluid From Sparse View Videos , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Yi Li,et al.  Data exploration of turbulence simulations using a database cluster , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[22]  Xiong Dun,et al.  Rainbow particle imaging velocimetry for dense 3D fluid velocity imaging , 2017, ACM Trans. Graph..

[23]  Chiara Bartolozzi,et al.  Event-Based Vision: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Bje Bert Blocken,et al.  PIV measurements of isothermal plane turbulent impinging jets at moderate Reynolds numbers , 2017 .

[25]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[26]  Marcus A. Magnor,et al.  Reconstructing the geometry of flowing water , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[27]  Hans-Peter Seidel,et al.  Time-resolved 3d capture of non-stationary gas flows , 2008, SIGGRAPH Asia '08.

[28]  Jingyi Yu,et al.  Reconstructing Gas Flows Using Light-Path Approximation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  G. McKinley,et al.  Springer Handbook of Experimental Fluid Mechanics , 2007 .

[30]  C. P. A C O R E T,et al.  Asynchronous event-based high speed vision for microparticle tracking , 2011 .

[31]  Huamin Wang,et al.  Physically guided liquid surface modeling from videos , 2009, ACM Trans. Graph..

[32]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[33]  T. Rösgen,et al.  Three-dimensional particle tracking velocimetry using dynamic vision sensors , 2017 .

[34]  Mark M. Weislogel,et al.  More investigations in capillary fluidics using a drop tower , 2013 .

[35]  Markus Raffel,et al.  Particle Image Velocimetry: A Practical Guide , 2002 .

[36]  N. Malik,et al.  Particle tracking velocimetry in three-dimensional flows , 1993 .

[37]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[38]  Andreas Schröder,et al.  From Noisy Particle Tracks to Velocity, Acceleration and Pressure Fields using B-splines and Penalties , 2016 .

[39]  T. Mcintyre,et al.  Magnetohydrodynamic drag force measurements in expansion tunnels using an accelerometer-based force balance , 2019, Experiments in Fluids.

[40]  Kostas Daniilidis,et al.  EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras , 2018, Robotics: Science and Systems.

[41]  Ajay K. Prasad,et al.  Stereoscopic particle image velocimetry , 2000 .

[42]  Vladlen Koltun,et al.  High Speed and High Dynamic Range Video with an Event Camera , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Paul E. Debevec,et al.  Acquisition of time-varying participating media , 2005, ACM Trans. Graph..

[44]  Brian S. Thurow,et al.  Time-Resolved 3D Flow-Measurement with a Single Plenoptic-Camera , 2019, AIAA Scitech 2019 Forum.

[45]  P. Lichtsteiner,et al.  Toward real-time particle tracking using an event-based dynamic vision sensor , 2011 .

[46]  W. Heidrich,et al.  Single-camera 3D PTV using particle intensities and structured light , 2019, Experiments in Fluids.

[47]  G. Settles,et al.  Schlieren “PIV” for turbulent flows , 2006 .

[48]  Chiara Bartolozzi,et al.  Event-driven ball detection and gaze fixation in clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[49]  Jun Sakakibara,et al.  High-speed scanning stereoscopic PIV for 3D vorticity measurement in liquids , 2004 .

[50]  Christoph Schnörr,et al.  Variational fluid flow measurements from image sequences: synopsis and perspectives , 2010 .

[51]  Sayan Biswas Schlieren Image Velocimetry (SIV) , 2018 .

[52]  Kevin F. Kelly,et al.  Hyperspectral Compressive Structured Light for Recovering Inhomogeneous Participating Media , 2017 .

[53]  Tobi Delbrück,et al.  Asynchronous Event-Based Binocular Stereo Matching , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Michael Hofstätter,et al.  A SPARC-compatible general purpose address-event processor with 20-bit l0ns-resolution asynchronous sensor data interface in 0.18μm CMOS , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[55]  Karl Krissian,et al.  A new energy-based method for 3D motion estimation of incompressible PIV flows , 2009, Comput. Vis. Image Underst..

[56]  C. Schnörr,et al.  Optical Stokes Flow Estimation: An Imaging‐Based Control Approach , 2006 .

[57]  Fulvio Scarano,et al.  Dense velocity reconstruction from tomographic PTV with material derivatives , 2016 .

[58]  Klaus D. Hinsch REVIEW ARTICLE: Holographic particle image velocimetry , 2002 .

[59]  A. Schröder,et al.  Shake-The-Box: Lagrangian particle tracking at high particle image densities , 2016, Experiments in Fluids.

[60]  Wolfgang Heidrich,et al.  Coupled Fluid Density and Motion from Single Views , 2018, Comput. Graph. Forum.

[61]  Tobi Delbruck,et al.  Robotic goalie with 3 ms reaction time at 4% CPU load using event-based dynamic vision sensor , 2013, Front. Neurosci..

[62]  Nicholas A. Worth,et al.  A laser sheet self-calibration method for scanning PIV , 2017 .

[63]  Konrad Schindler,et al.  3D Fluid Flow Estimation with Integrated Particle Reconstruction , 2018, International Journal of Computer Vision.

[64]  M. Gharib,et al.  Defocusing digital particle image velocimetry and the three-dimensional characterization of two-phase flows , 2002 .

[65]  T. Fahringer,et al.  Volumetric particle image velocimetry with a single plenoptic camera , 2015 .

[66]  Wolfgang Heidrich,et al.  Reconfigurable rainbow PIV for 3D flow measurement , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[67]  Julio Soria,et al.  High resolution volumetric dual-camera light-field PIV , 2019, Experiments in Fluids.

[68]  Kiriakos N. Kutulakos,et al.  Photo-Consistent Reconstruction of Semitransparent Scenes by Density-Sheet Decomposition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Marcus A. Magnor,et al.  Image-based tomographic reconstruction of flames , 2004, SIGGRAPH '04.

[70]  Chiara Bartolozzi,et al.  Event-Based Visual Flow , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[71]  Kostas Daniilidis,et al.  Event-based feature tracking with probabilistic data association , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[72]  Fulvio Scarano,et al.  Tomographic PIV: principles and practice , 2012 .

[73]  Ryad Benosman,et al.  Asynchronous event‐based high speed vision for microparticle tracking , 2012 .

[74]  Tobi Delbruck,et al.  A 240×180 10mW 12us latency sparse-output vision sensor for mobile applications , 2013, 2013 Symposium on VLSI Circuits.

[75]  Shigeru Nishio,et al.  Standard images for particle-image velocimetry , 2000 .

[76]  Bernhard Wieneke,et al.  Tomographic particle image velocimetry , 2006 .

[77]  Wolfgang Heidrich,et al.  From capture to simulation , 2014, ACM Trans. Graph..

[78]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[79]  Alexandra H. Techet,et al.  Three-dimensional synthetic aperture particle image velocimetry , 2010 .

[80]  M. Stöhr,et al.  Particle-tracking velocimetry , 1999 .

[81]  Hui Meng,et al.  Holographic particle velocimetry: a 3D measurement technique for vortex interactions, coherent structures and turbulence , 1991 .

[82]  Stefan Leutenegger,et al.  Simultaneous Optical Flow and Intensity Estimation from an Event Camera , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[83]  Davide Scaramuzza,et al.  EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time , 2017, International Journal of Computer Vision.

[84]  Ryad Benosman,et al.  Full-Field OCT Technique for High Speed Event-Based Optical Flow and Particle Tracking , 2018 .