The foundations of set theoretic estimation

Explains set theoretic estimation, which is governed by the notion of feasibility and produces solutions whose sole property is to be consistent with all information arising from the observed data and a priori knowledge. Each piece of information is associated with a set in the solution space, and the intersection of these sets, the feasibility set, represents the acceptable solutions. The practical use of the set theoretic framework stems from the existence of efficient techniques for finding these solutions. Many scattered problems in systems science and signal processing have been approached in set theoretic terms over the past three decades. The author synthesizes a single, general framework from these various approaches, examines its fundamental philosophy, goals, and analytical techniques, and relates it to conventional methods. >

[1]  J. Neumann On Rings of Operators. Reduction Theory , 1949 .

[2]  I. J. Schoenberg,et al.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[3]  S. Agmon The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[4]  Norbert Wiener,et al.  On the factorization of matrices , 1955 .

[5]  L. Zadeh What is optimal? (Edtl.) , 1958 .

[6]  W. Cheney,et al.  Proximity maps for convex sets , 1959 .

[7]  W. L. Miranker,et al.  The recovery of distorted band-limited signals , 1961 .

[8]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[9]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[10]  F. Schweppe Recursive state estimation: Unknown but bounded errors and system inputs , 1967 .

[11]  Fred C. Schweppe,et al.  The theory of amorphous cloud trajectory prediction , 1968, IEEE Trans. Inf. Theory.

[12]  H. Witsenhausen Sets of possible states of linear systems given perturbed observations , 1968 .

[13]  G. Herman,et al.  Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. , 1970, Journal of theoretical biology.

[14]  K. Tanabe Projection method for solving a singular system of linear equations and its applications , 1971 .

[15]  F. Schweppe,et al.  Control of linear dynamic systems with set constrained disturbances , 1971 .

[16]  D. Bertsekas,et al.  Recursive state estimation for a set-membership description of uncertainty , 1971 .

[17]  R. Gerchberg A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[18]  F. Schweppe,et al.  Continuous-time state estimation under disturbances bounded by convex sets , 1972 .

[19]  P. Gilbert Iterative methods for the three-dimensional reconstruction of an object from projections. , 1972, Journal of theoretical biology.

[20]  Thomas J. Rothenberg,et al.  Efficient estimation with a priori information , 1974 .

[21]  N. C. Gallagher,et al.  Method for Computing Kinoforms that Reduces Image Reconstruction Error. , 1973, Applied optics.

[22]  G. Herman,et al.  Three-dimensional reconstruction from projections: a review of algorithms. , 1974, International review of cytology.

[23]  R. Gerchberg Super-resolution through Error Energy Reduction , 1974 .

[24]  Frank Deutsch,et al.  Some Algorithms for Computing Best Approximations from Convex Cones , 1975 .

[25]  Gabor T. Herman,et al.  A relaxation method for reconstructing objects from noisy X-rays , 1975, Math. Program..

[26]  A. Papoulis A new algorithm in spectral analysis and band-limited extrapolation. , 1975 .

[27]  Guy Pierra,et al.  Eclatement de Contraintes en Parallèle pour la Minimisation d'une Forme Quadratique , 1975, Optimization Techniques.

[28]  Philip Wolfe,et al.  Finding the nearest point in A polytope , 1976, Math. Program..

[29]  Gabor T. Herman,et al.  Relaxation methods for image reconstruction , 1978, CACM.

[30]  Dante C. Youla,et al.  Generalized Image Restoration by the Method of Alternating Orthogonal Projections , 1978 .

[31]  Y. Censor,et al.  A New Approach to the Emission Computerized Tomography Problem: Simultaneous Calculation of Attenuation and Activity Coefficients , 1979, IEEE Transactions on Nuclear Science.

[32]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[33]  E. Fogel System identification via membership set constraints with energy constrained noise , 1979 .

[34]  Gabor T. Herman,et al.  Image reconstruction from projections : the fundamentals of computerized tomography , 1980 .

[35]  Didier Dubois,et al.  Fuzzy sets and systems ' . Theory and applications , 2007 .

[36]  Jean-Louis Goffin,et al.  The Relaxation Method for Solving Systems of Linear Inequalities , 1980, Math. Oper. Res..

[37]  D. Cahana,et al.  Restoration of arbitrary finite-energy optical objects from limited spatial and spectral information , 1981 .

[38]  A. Lent,et al.  An iterative method for the extrapolation of band-limited functions , 1981 .

[39]  Jean-Louis Goffin Convergence of a cyclic ellipsoid algorithm for systems of linear equalities , 1982, Math. Program..

[40]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[41]  M. Sezan,et al.  Image Restoration by the Method of Convex Projections: Part 2-Applications and Numerical Results , 1982, IEEE Transactions on Medical Imaging.

[42]  M. Hayes The reconstruction of a multidimensional sequence from the phase or magnitude of its Fourier transform , 1982 .

[43]  Yair Censor,et al.  Cyclic subgradient projections , 1982, Math. Program..

[44]  Y. Censor,et al.  New methods for linear inequalities , 1982 .

[45]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[46]  Ronald E. Bruck Random products of contractions in metric and Banach Spaces , 1982 .

[47]  Y. F. Huang,et al.  On the value of information in system identification - Bounded noise case , 1982, Autom..

[48]  M. Milanese,et al.  Estimation theory and uncertainty intervals evaluation in presence of unknown but bounded errors: Linear families of models and estimators , 1982 .

[49]  G. Herman Mathematical optimization versus practical performance: A case study based on the maximum entropy criterion in image reconstruction , 1982 .

[50]  W D Montgomery Optical applications of von Neumann's alternating-projection theorem. , 1982, Optics letters.

[51]  H. Stark,et al.  Signal restoration from phase by projections onto convex sets , 1983 .

[52]  Y. Censor,et al.  An interior points algorithm for the convex feasibility problem , 1983 .

[53]  H. Trussell,et al.  Convergence criteria for iterative restoration methods , 1983 .

[54]  Simeon Reich,et al.  A limit theorem for projections , 1983 .

[55]  J. N. Kapur Twenty-five years of maximum-entropy principle , 1983 .

[56]  Y. Censor,et al.  Strong underrelaxation in Kaczmarz's method for inconsistent systems , 1983 .

[57]  H Stark,et al.  Image restoration by convex projections in the presence of noise. , 1983, Applied optics.

[58]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[59]  Will Light,et al.  The Alternating Algorithm in Uniformly Convex Spaces , 1984 .

[60]  M. Sezan,et al.  Tomographic Image Reconstruction from Incomplete View Data by Convex Projections and Direct Fourier Inversion , 1984, IEEE Transactions on Medical Imaging.

[61]  Jan Mandel Convergence of the cyclical relaxation method for linear inequalities , 1984, Math. Program..

[62]  H. Elmikati,et al.  Extension of projection method to nonuniformly linear antenna arrays , 1984 .

[63]  H. Trussell,et al.  The feasible solution in signal restoration , 1984 .

[64]  Y. Censor Iterative Methods for the Convex Feasibility Problem , 1984 .

[65]  A. Abo-Taleb,et al.  Design of FIR two- dimensional digital filters by successive projections , 1984 .

[66]  Aharon Levi,et al.  Image restoration by the method of generalized projections with application to restoration from magnitude , 1984 .

[67]  Jae S. Lim,et al.  Signal reconstruction from Fourier transform sign information , 1985, IEEE Trans. Acoust. Speech Signal Process..

[68]  M. Arioli,et al.  A big-M type method for the computation of projections onto polyhedrons , 1985 .

[69]  B E Saleh,et al.  Image construction: optimum amplitude and phase masks in photolithography. , 1985, Applied optics.

[70]  A. Pierro,et al.  A simultaneous projections method for linear inequalities , 1985 .

[71]  A P Dhawan,et al.  Algorithms for limited-view computed tomography: an annotated bibliography and a challenge. , 1985, Applied optics.

[72]  Yih-Fang Huang,et al.  Asymptotically convergent modified recursive least-squares with data-dependent updating and forgetting factor , 1985, 1985 24th IEEE Conference on Decision and Control.

[73]  R. Marks,et al.  Signal synthesis in the presence of an inconsistent set of constraints , 1985 .

[74]  Roland T. Chin,et al.  Restoration of Multichannel Microwave Radiometric Images , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Didier Dubois,et al.  A review of fuzzy set aggregation connectives , 1985, Inf. Sci..

[76]  Bahaa E. A. Saleh,et al.  Image construction through diffraction-limited high-contrast imaging systems: an iterative approach , 1985 .

[77]  Yair Censor,et al.  An automatic relaxation method for solving interval linear inequalities , 1985 .

[78]  H. Trussell,et al.  The Landweber iteration and projection onto convex sets , 1985, IEEE Trans. Acoust. Speech Signal Process..

[79]  Yih-Fang Huang,et al.  A recursive estimation algorithm using selective updating for spectral analysis and adaptive signal processing , 1986, IEEE Trans. Acoust. Speech Signal Process..

[80]  D. Braess Nonlinear Approximation Theory , 1986 .

[81]  M. Reha Civanlar,et al.  Digital signal restoration using fuzzy sets , 1986, IEEE Trans. Acoust. Speech Signal Process..

[82]  D. Youla,et al.  Extensions of a result on the synthesis of signals in the presence of inconsistent constraints , 1986 .

[83]  B. Efron Why Isn't Everyone a Bayesian? , 1986 .

[84]  G. T. Poulton Antenna power pattern synthesis using method of successive projections , 1986 .

[85]  H. Trussell,et al.  Constructing membership functions using statistical data , 1986 .

[86]  A. Kandel Fuzzy Mathematical Techniques With Applications , 1986 .

[87]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[88]  Alfredo N. Iusem,et al.  Convergence results for an accelerated nonlinear cimmino algorithm , 1986 .

[89]  Osami Sasaki,et al.  Image restoration in singular vector space by the method of convex projections. , 1987, Applied optics.

[90]  H. Trussell,et al.  Errors in Reprojection Methods in Computenzed Tomography , 1987, IEEE Transactions on Medical Imaging.

[91]  J. P. Norton,et al.  Identification and application of bounded-parameter models , 1985, Autom..

[92]  M. Sezan,et al.  Incorporation of a priori moment information into signal recovery and synthesis problems , 1987 .

[93]  Michel Bouvet,et al.  Constrained Wiener filtering , 1987, IEEE Trans. Inf. Theory.

[94]  E. Veklerov,et al.  Stopping Rule for the MLE Algorithm Based on Statistical Hypothesis Testing , 1987, IEEE Transactions on Medical Imaging.

[95]  Gordon G. Johnson A nonconvex set which has the unique nearest point property , 1987 .

[96]  José María Carazo,et al.  Information recovery in missing angular data cases: an approach by the convex projections method in three dimensions , 1987 .

[97]  Robert J. Marks Class of continuous level associative memory neural nets , 1987 .

[98]  R. Tempo Robust estimation and filtering in the presence of bounded noise , 1987, 26th IEEE Conference on Decision and Control.

[99]  M. Reha Civanlar,et al.  Error bounds for iterative reprojection methods in computerized tomography , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[100]  Ahmet Enis Cetin,et al.  Iterative procedure for designing two-dimensional FIR filters , 1987 .

[101]  PETER SANTAGO,et al.  Using Convex Set Techniques for Combined Pixel and Frequency Domain Coding of Time-Varying Images , 1987, IEEE J. Sel. Areas Commun..

[102]  Jan P. Allebach,et al.  Iterative reconstruction of bandlimited images from nonuniformly spaced samples , 1987 .

[103]  S Ebstein,et al.  Stellar speckle interferometry energy spectrum recovery by convex projections. , 1987, Applied optics.

[104]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[105]  Patrice Y. Simard,et al.  A Projection Operator for the Restoration of Divergence-Free Vector Fields , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[106]  A. Pierro,et al.  A finitely convergent “row-action” method for the convex feasibility problem , 1988 .

[107]  R. Kosut Adaptive control via parameter set estimation , 1988 .

[108]  Rashid Ansari,et al.  Convolution-based framework for signal recovery and applications , 1988 .

[109]  N. Ottavy Strong convergence of projection-like methods in Hilbert spaces , 1988 .

[110]  H. Stark,et al.  Tomographic image reconstruction using the theory of convex projections. , 1988, IEEE transactions on medical imaging.

[111]  Yair Censor,et al.  Parallel application of block-iterative methods in medical imaging and radiation therapy , 1988, Math. Program..

[112]  Ronald K. Pearson,et al.  Block-sequential algorithms for set-theoretic estimation , 1988 .

[113]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[114]  Howard L. Weinert,et al.  Error bounds for the method of alternating projections , 1988, Math. Control. Signals Syst..

[115]  Poorvi L. Vora,et al.  Bounds on restoration quality using a priori information , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[116]  M. R. Civanlar,et al.  Optimal pulse shape design using projections onto convex sets , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[117]  Harold A. Sabbagh,et al.  An eddy-current model and algorithm for three-dimensional nondestructive evaluation of advanced composites , 1988 .

[118]  J. R. Deller,et al.  Linear prediction analysis of speech based on set-membership theory , 1989 .

[119]  J Llacer,et al.  Feasible images and practical stopping rules for iterative algorithms in emission tomography. , 1989, IEEE transactions on medical imaging.

[120]  P. Simard,et al.  Restoration of the velocity field of the heart from two-dimensional echocardiograms. , 1989, IEEE transactions on medical imaging.

[121]  Karel J. Keesman,et al.  Identification and prediction propagation of uncertainty in models with bounded noise , 1989 .

[122]  James A. Cadzow,et al.  Algebraic approach to two-dimensional recursive digital filter synthesis , 1989, IEEE Trans. Acoust. Speech Signal Process..

[123]  Robert J. Marks,et al.  Alternating projection neural networks , 1989 .

[124]  Y. Censor,et al.  Block-iterative projection methods for parallel computation of solutions to convex feasibility problems , 1989 .

[125]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[126]  E. Walter,et al.  Exact recursive polyhedral description of the feasible parameter set for bounded-error models , 1989 .

[127]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[128]  J.R. Deller A 'systolic array' formulation of the optimal bounding ellipsoid algorithm , 1989, IEEE Trans. Acoust. Speech Signal Process..

[129]  J. Deller Set membership identification in digital signal processing , 1989, IEEE ASSP Magazine.

[130]  R. Mathar,et al.  A cyclic projection algorithm via duality , 1989 .

[131]  Ronald E. Burge,et al.  Two optimisation approaches to COHOE design , 1989 .

[132]  S. Pei,et al.  Design of a class of time-constrained FIR digital filters by successive projections , 1989 .

[133]  H. Stark,et al.  A comparative study of three reconstruction methods for a limited-view computer tomography problem. , 1989, IEEE transactions on medical imaging.

[134]  Patrick L. Combettes,et al.  Methods for digital restoration of signals degraded by a stochastic impulse response , 1989, IEEE Trans. Acoust. Speech Signal Process..

[135]  Eric Walter,et al.  Characterization of non-connected parameter uncertainty regions , 1990 .

[136]  H. Trussell,et al.  Method of successive projections for finding a common point of sets in metric spaces , 1990 .

[137]  A. Murat Tekalp,et al.  Adaptive image restoration with artifact suppression using the theory of convex projections , 1990, IEEE Trans. Acoust. Speech Signal Process..

[138]  Patrice Y. Simard,et al.  Vector field restoration by the method of convex projections , 1990, Comput. Vis. Graph. Image Process..

[139]  J. Zowe,et al.  Relaxed outer projections, weighted averages and convex feasibility , 1990 .

[140]  B. Yegnanarayana,et al.  Image reconstruction from noisy digital holograms , 1990 .

[141]  Messaoud Benidir,et al.  Nonconvexity of the stability domain of digital filters , 1990, IEEE Trans. Acoust. Speech Signal Process..

[142]  Yih-Fang Huang,et al.  ARMA parameter estimation using a novel recursive estimation algorithm with selective updating , 1990, IEEE Trans. Acoust. Speech Signal Process..

[143]  M. Ibrahim Sezan,et al.  Survey of recent developments in digital image restoration. , 1990 .

[144]  Henry Stark,et al.  Iterative and one-step reconstruction from nonuniform samples by convex projections , 1990 .

[145]  H Stark,et al.  Projection method formulations of Hopfield-type associative memory neural networks. , 1990, Applied optics.

[146]  Giorgio Franceschetti,et al.  Intersection approach to array pattern synthesis , 1990 .

[147]  A. Oppenheim,et al.  Reconstruction of two-dimensional signals from level crossings , 1990 .

[148]  M. R. Civanlar,et al.  Comments on "Design of a class of time-constrained FIR digital filters by successive projections , 1990 .

[149]  Aggelos K. Katsaggelos,et al.  New termination rule for linear iterative image restoration algorithms , 1990 .

[150]  H. Trussell,et al.  Set theoretic autoregressive spectral estimation , 1990, Fifth ASSP Workshop on Spectrum Estimation and Modeling.

[151]  G. Crombez Image Recovery by Convex Combinations of Projections , 1991 .

[152]  Alvaro R. De Pierro,et al.  On the convergence of Han's method for convex programming with quadratic objective , 1991, Math. Program..

[153]  Langford B. White,et al.  Signal synthesis from Cohen's class of bilinear time-frequency signal representations using convex projections , 1991, IEEE International Conference on Acoustics, Speech, and Signal Processing.

[154]  Darryl Morrell,et al.  Set-values filtering and smoothing , 1991, IEEE Trans. Syst. Man Cybern..

[155]  N. Hurt Signal enhancement and the method of successive projections , 1991 .

[156]  Patrick L. Combettes,et al.  The foundations of set theoretic estimation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[157]  J Shamir,et al.  Application of the projection-onto-constraint-sets algorithm for optical pattern recognition. , 1991, Optics letters.

[158]  Patrick L. Combettes,et al.  Set theoretic estimation by random search , 1991, IEEE Trans. Signal Process..

[159]  H. Trussell Applications of set theoretic methods to color systems , 1991 .

[160]  Aggelos K. Katsaggelos,et al.  A regularized iterative image restoration algorithm , 1991, IEEE Trans. Signal Process..

[161]  Patrick L. Combettes,et al.  The use of noise properties in set theoretic estimation , 1991, IEEE Trans. Signal Process..

[162]  A. Murat Tekalp,et al.  Comparative study of some statistical and set-theoretic methods for image restoration , 1991, CVGIP Graph. Model. Image Process..

[163]  Peter Kosmol,et al.  The product of affine orthogonal projections , 1991 .

[164]  M. Ibrahim Sezan,et al.  Prototype image constraints for set-theoretic image restoration , 1991, IEEE Trans. Signal Process..

[165]  Ovidio Mario Bucci,et al.  Reconfigurable arrays by phase-only control , 1991 .

[166]  M Hedley,et al.  Motion artifact correction in MRI using generalized projections. , 1991, IEEE transactions on medical imaging.

[167]  A. Enis Çetin,et al.  An iterative algorithm for signal reconstruction from bispectrum , 1991, IEEE Trans. Signal Process..

[168]  I. Csiszár Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[169]  Darryl Morrell,et al.  Convex Bayes decision theory , 1991, IEEE Trans. Syst. Man Cybern..

[170]  J. R. Cruz,et al.  Enhanced autoregressive moving average spectral estimation applied to the measurement of Doppler spectral width , 1991, IEEE Trans. Geosci. Remote. Sens..

[171]  Stéphane Mallat,et al.  Zero-crossings of a wavelet transform , 1991, IEEE Trans. Inf. Theory.

[172]  Henry Stark,et al.  Design of phase gratings by generalized projections , 1991 .

[173]  William Menke,et al.  Applications of the POCS inversion method to interpolating topography and other geophysical fields , 1991 .

[174]  G. T. Poulton,et al.  Efficient design of shaped reflectors using successive projections , 1991 .

[175]  Anne M. Landraud Image restoration and enhancement of characters, using convex projection methods , 1991, CVGIP Graph. Model. Image Process..

[176]  Martin Vetterli,et al.  Optimal MSE signal reconstruction in oversampled A/D conversion using convexity , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[177]  K. G. Murty,et al.  New iterative methods for linear inequalities , 1992 .

[178]  Patrick L. Combettes,et al.  A general framework for the incorporation of uncertainty in set theoretic estimation , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[179]  Richard J. Mammone,et al.  Image restoration by convex projections using adaptive constraints and the L1 norm , 1992, IEEE Trans. Signal Process..

[180]  Langford B. White,et al.  The wide-band ambiguity function and Altes' Q-distribution: Constrained synthesis and time-scale filtering , 1992, IEEE Trans. Inf. Theory.

[181]  Avideh Zakhor,et al.  Reconstruction of oversampled band-limited signals from Sigma Delta encoded binary sequences , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[182]  C T Chen,et al.  Superresolved tomography by convex projections and detector motion. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[183]  Patrick L. Combettes,et al.  Best stable and invertible approximations for ARMA systems , 1992, IEEE Trans. Signal Process..

[184]  Edward L. Titlebaum,et al.  Estimation of the parameters of a multipath channel using set-theoretic deconvolution , 1992, IEEE Trans. Commun..

[185]  Hiroyuki Kudo,et al.  Sinogram recovery with the method of convex projections for limited-data reconstruction in computed tomography , 1992 .

[186]  Henry Stark,et al.  Projection-based image restoration , 1992 .

[187]  Irwin E. Schochetman,et al.  Convergence of best approximations from unbounded sets , 1992 .

[188]  Kuansan Wang,et al.  Auditory representations of acoustic signals , 1992, IEEE Trans. Inf. Theory.

[189]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[190]  Patrick L. Combettes,et al.  Parallel projection methods for set theoretic signal reconstruction and restoration , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[191]  J. R. Deller,et al.  Least-square identification with error bounds for real-time signal processing and control , 1993, Proc. IEEE.

[192]  Patrick L. Combettes,et al.  Signal recovery by best feasible approximation , 1993, IEEE Trans. Image Process..

[193]  Avideh Zakhor,et al.  Reconstruction of oversampled band-limited signals from ΣΔ encoded binary sequences , 1994, IEEE Trans. Signal Process..