Unified reconstruction of RAW HDR video data

Traditional high dynamic range (HDR) capture has mostly relied on merging images captured with different exposure times. While this works well for static scenes, dynamic scenes pose difficult challenges because registration of differently exposed images often leads to ghosting and other artifacts. This chapter reviews methods which capture HDR video frames within a single exposure time, using either multiple synchronized sensors or multiplexing of the sensor response spatially across the sensor. Most previous HDR reconstruction methods perform demosaicing, noise reduction, resampling (registration), and HDR fusion in separate steps. This chapter presents a framework for unified HDR reconstruction, including all steps in the traditional imaging pipeline in a single adaptive filtering operation, and describes an image formation model and a sensor noise model applicable to both single-sensor and multisensor systems. The benefits of using raw data directly are demonstrated with examples using input data from multiple synchronized sensors, and single images with varying per-pixel gain.