High-resolution images from a sequence of low-resolution observations

This chapter discusses the problem of obtaining a high resolution (HR) image or sequences of HR images from a set of low resolution (LR) observations. This problem has also been referred to in the literature by the names of super-resolution (SR) and resolution enhancement (we will be using all terms interchangeably). These LR images are under-sampled and they are acquired by either multiple sensors imaging a single scene or by a single sensor imaging the scene over a period of time. For static scenes the LR observations are related by global subpixel shifts, while for dynamic scenes they are related by local subpixel shifts due to object motion (camera motion, such as panning and zooming can also be included in this model). In this chapter we will be using the terms image(s), image frame(s), and image sequence frame(s) interchangeably. This is a problem encountered in a plethora of applications. Images and video of higher and higher resolution are required, for example, in scientific (e.g., medical, space exploration, surveillance) and commercial (e.g., entertainment, high-definition television) applications. One of the early applications of high resolution imaging was with Landsat imagery. The orbiting satellite would go over the same area every 18 days, acquiring misregistered images. Appropriately combining these LR images produced HR images of the scene. Increasing the resolution of the imaging sensor is clearly one way to increase the resolution of the acquired images. This solution however may not be feasible due to the increased associated cost and the fact that the shot noise increases during acquisition as the pixel size becomes smaller. On the other hand, increasing the chip size to accommodate the larger number of