A Supervised Learning Approach for Dynamic Sampling

Sparse sampling schemes have the potential to reduce image acquisition time by reconstructing a desired image from a sparse subset of measured pixels. Moreover, dynamic sparse sampling methods have the greatest potential because each new pixel is selected based on information obtained from previous samples. However, existing dynamic sampling methods tend to be computationally expensive and therefore too slow for practical application. In this paper, we present a supervised learning based algorithm for dynamic sampling (SLADS) that uses machine-learning techniques to select the location of each new pixel measurement. SLADS is fast enough to be used in practical imaging applications because each new pixel location is selected using a simple regression algorithm. In addition, SLADS is accurate because the machine learning algorithm is trained using a total reduction in distortion metric which accounts for distortion in a neighborhood of the pixel being sampled. We present results on both computationally-generated synthetic data and experimentallycollected data that demonstrate substantial improvement relative to state-of-the-art static sampling methods. Introduction In conventional point-wise image acquisition, all pixels in a rectilinear grid are measured. However, in many imaging applications, a high-fidelity pixel measurement could take up to 1 second. Examples of such methods include electron back scatter diffraction (EBSD) microscopy and Raman spectroscopy, which are of great importance in material science and chemistry [1]. Then, acquiring a complete set of high-resolution measurements on these imaging applications becomes impractical. Sparse sampling offers the potential to dramatically reduce the time required to acquire an image. In this approach, a sparse set of pixels is measured, and the full resolution image is reconstructed from the set of sparse measurements. In addition to speeding image acquisition, sparse sampling methods also hold the potential to reduce the exposure of the object being imaged to destructive radiation. This is of critical importance when imaging biological samples using X-rays, electrons, or even optical photons [2, 3]. Sparse sampling approaches fall into two main categories: static and dynamic. In static sampling, pixels are measured in a pre-defined order. Examples of static sparse sampling methods include random sampling strategies such as in [4], and lowdiscrepancy sampling [5]. As a result some samples from these methods may not be very informative, as they do not take into account the object being scanned. There are static sampling methods based on an a priori knowledge of the object geometry and sparsity such as [6, 7]. However a priori knowledge is not always available for general imaging applications On the other hand, dynamic sampling (DS) methods adaptively determine new measurement locations based on the information obtained from previous measurements. This is a very powerful technique since in real applications previous measurements can tell one a great deal about the object being scanned and also about the best locations for future measurements Therefore, dynamic sampling has the potential to dramatically reduce the total number of samples required to achieve a particular level of distortion in the reconstructed image. An example of a dynamic sampling method was proposed in [8] by Kovačević et al. Here initially an object is measured with a sparse grid. Then, if the intensity of a pixel is above a certain threshold, the vicinity of that pixel is measured in higher resolution. However, the threshold was empirically chosen for the specific scanner and thus this method cannot be generalized for different imaging modalities. For general applications, a set of DS methods has been proposed in previous literature where an objective function is designed and the measurements are chosen to optimize that objective function. For instance, dynamic compressive sensing methods [9–11] find the next measurements that maximally reduces the differential entropy. However, dynamic compressive sensing methods use an unconstrained projection as a measurement and therefore are not suitable for point-wise measurements where the measurement is constrained. Apart from these methods, application specific DS methods that optimize an objective function to find the next measurement have been developed. One example is [12], where the authors modify the optimal experimental design [13] framework to incorporate dynamic measurement selection in a biochemical network. Seeger et al. in [14] also finds the measurement that reduces the differential entropy the most but now to select optimal K-space spiral and line measurements for magnetic resonance imaging (MRI). In addition, Batenburg et al. [15] propose a DS method for binary computed tomography in which the measurement that maximizes the information gain is selected. Even though these measurements are constrained they are application specific and therefore not applicable to general point-wise measurements. In [16] Godaliyadda et al. propose a DS algorithm for general point-wise measurements. Here, the authors use a MonteCarlo simulation method to approximate the conditional variance at every unmeasured location, given previous measurements, and select the pixel with largest conditional variance. However, Monte-Carlo simulation methods such as the Metropolis-Hastings method are very slow and therefore this method is infeasible for real-time applications. Furthermore, the objective function in this method does not account for the change of conditional variance in ©2016 Society for Imaging Science and Technology DOI: 10.2352/ISSN.2470-1173.2016.19.COIMG-153 IS&T International Symposium on Electronic Imaging 2016 Computational Imaging XIV COIMG-153.1 the entire image with a new measurement. In this paper, we propose a new DS algorithm for point-wise measurements named supervised learning approach for dynamic sampling (SLADS). The objective of SLADS is to select a new pixel so as to maximally reduce the conditional expectation of the reduction in distortion (ERD) in the entire reconstructed image. In SLADS, we compute the reduction in distortion for each pixel in a training data set, and then find the relationship between the ERD and a local feature vector through a regression algorithm. Since we use a supervised learning approach, we can very rapidly estimate the ERD at each pixel in the unknown testing image. Moreover, we introduce a measure that approximates the distortion reduction in the training dataset so that it accounts for the distortion reduction in the pixel and its neighbors. Since computing the distortion reduction for each pixel during training can be intractable, particularly for large images, this approximation is vital to make the training procedure feasible. Experimental results on sampling a computationally-generated synthetic EBSD image and an experimentally-collected image have shown that SLADS can compute a new sample locations very quickly (in the range of 5 500 ms), and can achieve the same reconstruction distortion as static sampling methods with dramatically fewer samples (2-4 times fewer). Dynamic Sampling Framework The objective in sparse sampling is to measure a sparse set of pixels in an image and then reconstruct the full resolution image from those sparse samples. Moreover, with sparse dynamic sampling, the location for each new pixel to measure will be informed by all the previous pixel measurements. To formulate the problem, we denote the image we would like to measure as X ∈ RN , where Xr is a pixel at location r ∈ Ω. Furthermore, let us assume that k pixels have been measured at a set of locations S = {s(1), · · · ,s(k)}, and that the corresponding measured values and locations are represented by the k×2 matrix Y (k) =  s,Xs(1) .. s,Xs(k)  . Then from Y (k), we can reconstruct an image X̂ (k), which is our best estimate of X given the first k measurements. Now, if we select Xs as our next pixel to measure, then presumably we can reconstruct a better estimate of the image, which we will denote by X̂ (k;s). So then X̂ (k;s) is our best estimate of X given both Y (k) and Xs. So at this point, our goal is to select the next location s(k+1) that results in the greatest decrease in reconstruction distortion. In order to formulate this problem, let D(Xr, X̂r) denote the distortion measure between a pixel Xr and its estimate X̂r, and let D(X , X̂) = ∑ r∈Ω D(Xr, X̂r) , (1) denote the total distortion between the image X and its estimate X̂ . Then using this notation, we may define R r to be the local reduction in distortion at pixel r that would result from the measurement of the pixel Xs. R r = D(Xr, X̂ (k) r )−D(Xr, X̂ (k;s) r ) (2) Importantly, the measurement of the pixel Xs does not only reduce distortion at that pixel. It also reduces the distortion at neighboring pixels. So in order to represent the total reduction in distortion, we must sum over all pixels r ∈Ω. R(k;s) = ∑ r∈Ω R r (3) = D(X , X̂ (k))−D(X , X̂ (k;s)) . (4) Now of course, we do not know what the value of Xs until it is measured; so we also do not know the value R(k;s). Therefore, we must make our selection of the next pixel based on the conditional expectation of reduction in distortion which we will refer to as the ERD given by R̄(k;s) = E [ R(k;s)|Y (k) ] . (5) So with this notation, our goal is to efficiently compute the next pixel to sample, s(k+1), as the solution to the following optimization. s(k+1) = arg max s∈{Ω\S } ( R̄(k;s) ) (6) Once we measure the location Xs(k+1) , then we form the new measurement vector Y (k+1) = [ Y (k) s,Xs(k+1) ]