Real-time iterative framework of regularized image restoration and its application to video enhancement

A novel framework of real-time video enhancement is proposed. The proposed framework is based on the regularized iterative image restoration algorithm, which iteratively removes degradation effects under a priori constraints. Although regularized iterative image restoration is proven to be a successful technique in restoring degraded images, its application is limited within still images or off-line video enhancement because of its iterative structure. In order to enable this iterative restoration algorithm to enhance the quality of video in real-time, each frame of video is considered as the constant input and the processed previous frame is considered as the previous iterative solution. This modification is valid only when the input of the iteration, that is each frame, remains unchanged throughout the iteration procedure. Because every frame of general video sequence is different from each other, each frame is segmented into two regions: still background and moving objects. These two regions are processed differently by using a segmentation-based spatially adaptive restoration and a background generation algorithms. Experimental results show that the proposed real-time restoration algorithm can enhance the input video much better than simple filtering techniques. The proposed framework enables real-time video enhancement at the cost of image quality only in the moving object area of dynamic shots, which is relatively insensitive to the human visual system.