Emulating displays with continuously varying frame rates

The visual quality of a motion picture is significantly influenced by the choice of the presentation frame rate. Increasing the frame rate improves the clarity of the image and helps to alleviate many artifacts, such as blur, strobing, flicker, or judder. These benefits, however, come at the price of losing well-established film aesthetics, often referred to as the "cinematic look". Current technology leaves artists with a sparse set of choices, e.g., 24 Hz or 48 Hz, limiting the freedom in adjusting the frame rate to artistic needs, content, and display technology. In this paper, we solve this problem by proposing a novel filtering technique which enables emulating the whole spectrum of presentation frame rates on a single-frame-rate display. The key component of our technique is a set of simple yet powerful filters calibrated and evaluated in psychophysical experiments. By varying their parameters we can achieve an impression of continuously varying presentation frame rate in both the spatial and temporal dimensions. This allows artists to achieve the best balance between the aesthetics and the objective quality of the motion picture. Furthermore, we show how our technique, informed by cinematic guidelines, can adapt to the content and achieve this balance automatically.

[1]  SeidelHans-Peter,et al.  Emulating displays with continuously varying frame rates , 2016 .

[2]  Xiao-Fan Feng LCD motion-blur analysis, perception, and reduction using synchronized backlight flashing , 2006, Electronic Imaging.

[3]  Frédo Durand,et al.  Antialiasing for automultiscopic 3D displays , 2006, EGSR '06.

[4]  H DE LANGE DZN,et al.  Research into the dynamic nature of the human fovea-cortex systems with intermittent and modulated light. II. Phase shift in brithtness and delay in color perception. , 1958, Journal of the Optical Society of America.

[5]  Yuichi Taguchi,et al.  Motion-Aware Structured Light Using Spatio-Temporal Decodable Patterns , 2012, ECCV.

[6]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[7]  Andrew B. Watson,et al.  High Frame Rates and Human Vision: A View through the Window of Visibility , 2013 .

[8]  Jérôme Théau,et al.  Temporal Resolution , 2008, Encyclopedia of GIS.

[9]  Kees Teunissen,et al.  Perceived sharpness in complex moving images , 1995 .

[10]  Ashok Veeraraghavan,et al.  Towards Motion Aware Light Field Video for Dynamic Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  David Whitaker,et al.  Effects of luminance and external temporal noise on flicker sensitivity as a function of stimulus size at various eccentricities , 1994, Vision Research.

[12]  H. D. L. Dzn Research into the dynamic nature of the human fovea-cortex systems with intermittent and modulated light. I. Attenuation characteristics with white and colored light. , 1958 .

[13]  Francisco J. Serón,et al.  Perceptual considerations for motion blur rendering , 2011, TAP.

[14]  H DE LANGE DZN,et al.  Research into the dynamic nature of the human fovea-cortex systems with intermittent and modulated light. I. Attenuation characteristics with white and colored light. , 1958, Journal of the Optical Society of America.

[15]  Martin S Banks,et al.  Temporal presentation protocols in stereoscopic displays: Flicker visibility, perceived motion, and perceived depth , 2011, Journal of the Society for Information Display.

[16]  David Stump Frame Rates and Aspect Ratios , 2014 .

[17]  Ramesh Raskar,et al.  Reinterpretable Imager: Towards Variable Post‐Capture Space, Angle and Time Resolution in Photography , 2010, Comput. Graph. Forum.

[18]  Anne Sullivan,et al.  Synthetic Shutter Speed Imaging , 2007, Comput. Graph. Forum.

[19]  George Drettakis,et al.  Perception of Visual Artifacts in Image‐Based Rendering of Façades , 2011, EGSR '11.

[20]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, TAP.

[21]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[22]  Kevin Brownlow,et al.  Silent Films–What Was the Right Speed? , 1990 .

[23]  Barry Salt,et al.  Film Style and Technology: History and Analysis , 1983 .

[24]  Hung-Chi Lee,et al.  Humans perceive flicker artifacts at 500 Hz , 2015, Scientific Reports.

[25]  Kelvin Sung,et al.  Spatial-Temporal Antialiasing , 2002, IEEE Trans. Vis. Comput. Graph..

[26]  Scott Daly,et al.  Engineering observations from spatiovelocity and spatiotemporal visual models , 1998, Electronic Imaging.

[27]  Scott Daly,et al.  A Psychophysical Study Exploring Judder Using Fundamental Signals and Complex Imagery , 2014 .

[28]  Hans-Peter Seidel,et al.  Real-time temporal shaping of high-speed video streams , 2010, Comput. Graph..

[29]  Ashok Veeraraghavan,et al.  Flexible Voxels for Motion-Aware Videography , 2010, ECCV.

[30]  잭슨 바나비,et al.  Method and apparatus for photographing and projecting moving images , 2010 .

[31]  Hans-Peter Seidel,et al.  Perceptually‐motivated Real‐time Temporal Upsampling of 3D Content for High‐refresh‐rate Displays , 2010, Comput. Graph. Forum.

[32]  Irfan A. Essa,et al.  Image-based motion blur for stop motion animation , 2001, SIGGRAPH.

[33]  David Richards 120fps as a Universal Production Format for Motion Pictures , 2014 .

[34]  Marcus A. Magnor,et al.  Temporal Video Filtering and Exposure Control for Perceptual Motion Blur , 2015, IEEE Transactions on Visualization and Computer Graphics.

[35]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[36]  Robert S. Allison,et al.  Evidence that Viewers Prefer Higher Frame-Rate Film , 2015, ACM Trans. Appl. Percept..