Coded Strobing Photography : Compressive Sensing of High-speed Periodic Events

We show that, via temporal modulation, one can observe a high-speed periodic event well beyond the abiliti es of a low-frame rate camera. By strobing the exposure with unique sequences within the integration time of each frame, we take coded projections of dynamic events. From a sequence of such frames, we reconstruct a high-speed video of the high frequency periodic process. Strobing is used in entertainment, medical imaging and industrial inspection to generate lowe r beat frequencies. But this is limited to scenes with a detect able single dominant frequency and requires high-intensity lighting. In this paper, we address the problem of sub-Nyquist samplin g of periodic signals and show designs to capture and reconstruc t such signals. The key result is that for such signals the Nyquist r ate constraint can be imposed on strobe-rate rather than the sen sorrate. The technique is based on intentional aliasing of the f requency components of the periodic signal while the reconstr uction algorithm exploits recent advances in sparse representati ons and compressive sensing. We exploit the sparsity of periodic si gnals in Fourier domain to develop reconstruction algorithms that are inspired by compressive sensing.

[1]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[2]  Ivars Bilinskis,et al.  Randomized Signal Processing , 1992 .

[3]  Andrew Blake,et al.  Motion Deblurring and Super-resolution from an Image Sequence , 1996, ECCV.

[4]  Brian A. Wandell,et al.  Multiple Capture Single Image Architecture with a CMOS Sensor , 1999 .

[5]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  H. Larsson,et al.  Vocal Fold Vibrations: High‐Speed Imaging, Kymography, and Acoustic Analysis: A Preliminary Report , 2000, The Laryngoscope.

[7]  I R Titze,et al.  Irregular vocal-fold vibration--high-speed observation and modeling. , 2000, The Journal of the Acoustical Society of America.

[8]  Hayit Greenspan,et al.  MRI Inter-slice Reconstruction Using Super-Resolution , 2001, MICCAI.

[9]  F. Marvasti Nonuniform sampling : theory and practice , 2001 .

[10]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[11]  M Hess,et al.  [Physical and technical elements of short-interval, color-filtered double strobe flash-stroboscopy]. , 2002, HNO.

[12]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[13]  Fabrice Labeau,et al.  Discrete Time Signal Processing , 2004 .

[14]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[15]  Zhouchen Lin,et al.  Response to the Comments on "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation' , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Serge J. Belongie,et al.  Structure from Periodic Motion , 2004, SCVMA.

[17]  Shree K. Nayar,et al.  Motion-based motion deblurring , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Steven M. Seitz,et al.  View-Invariant Analysis of Cyclic Motion , 1997, International Journal of Computer Vision.

[19]  Hans-Peter Seidel,et al.  Pitching a baseball: tracking high-speed motion with multi-exposure images , 2004, ACM Trans. Graph..

[20]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[21]  Patrick Pérez,et al.  Periodic motion detection and segmentation via approximate sequence alignment , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[23]  M. Gustafsson Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Amit K. Agrawal,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, ACM Trans. Graph..

[25]  Tai-Pang Wu,et al.  Video repairing under variable illumination using cyclic motions , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  E.J. Candes Compressive Sampling , 2022 .

[27]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[28]  Dimitar D Deliyski,et al.  Mucosal wave: a normophonic study across visualization techniques. , 2008, Journal of voice : official journal of the Voice Foundation.

[29]  Frédo Durand,et al.  Motion-invariant photography , 2008, ACM Trans. Graph..

[30]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[31]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[32]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[33]  C. Webster Animals in Motion , 2012 .