Distributed redundant representations in man-made and biological sensing systems

Ph.D., Electrical and Computer Engineering Rice University May 2007 Thesis: Distributed redundant representations in man-made and biological sensing systems Advisor: Dr. Don H. Johnson M.S., Electrical Engineering Rice University May 2002 B.S.E., Computer Engineering (magna cum laude) University of Michigan — Ann Arbor April 2000 B.F.A., Performing Arts Technology — Music University of Michigan — Ann Arbor April 2000

[1]  Richard G. Baraniuk,et al.  Locally Competitive Algorithms for Sparse Approximation , 2007, 2007 IEEE International Conference on Image Processing.

[2]  Venkat Chandrasekaran,et al.  Representation and Compression of Multidimensional Piecewise Functions Using Surflets , 2009, IEEE Transactions on Information Theory.

[3]  Peter G. Casazza,et al.  Modeling sensor networks with fusion frames , 2007, SPIE Optical Engineering + Applications.

[4]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[5]  Andrea J. Goldsmith,et al.  Energy-constrained modulation optimization , 2005, IEEE Transactions on Wireless Communications.

[6]  Shie Qian,et al.  Signal representation using adaptive normalized Gaussian functions , 1994, Signal Process..

[7]  Joel A. Tropp,et al.  Sparse Approximation Via Iterative Thresholding , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[8]  R. Fay,et al.  The Mammalian auditory pathway : neurophysiology , 1992 .

[9]  Don H. Johnson,et al.  Examining methods for estimating mutual information in spiking neural systems , 2005, Neurocomputing.

[10]  P. Casazza THE ART OF FRAME THEORY , 1999, math/9910168.

[11]  Vivek K Goyal,et al.  Quantized Frame Expansions with Erasures , 2001 .

[12]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[13]  T. Berger Rate-Distortion Theory , 2003 .

[14]  D.H. Johnson,et al.  Power Scheduling for Wireless Sensor and Actuator Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[15]  Ivan W. Selesnick,et al.  The design of approximate Hilbert transform pairs of wavelet bases , 2002, IEEE Trans. Signal Process..

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  Don H. Johnson,et al.  Information Theory and Neuroscience: A Tutorial , 2006 .

[18]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[19]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[20]  Helmut Bölcskei,et al.  Frame-theoretic analysis of oversampled filter banks , 1998, IEEE Trans. Signal Process..

[21]  Andrea J. Goldsmith,et al.  Energy-efficient joint estimation in sensor networks: analog vs. digital , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[22]  Koen Langendoen,et al.  Distributed localization in wireless sensor networks: a quantitative compariso , 2003, Comput. Networks.

[23]  N.J. Halas,et al.  All-optical nanoscale pH meter: a plasmonic nanodevice with quantifiable output , 2006, LEOS 2006 - 19th Annual Meeting of the IEEE Lasers and Electro-Optics Society.

[24]  David R. Cox,et al.  The statistical analysis of series of events , 1966 .

[25]  Ian F. Akyildiz,et al.  Wireless sensor and actor networks: research challenges , 2004, Ad Hoc Networks.

[26]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[27]  Yinyu Ye,et al.  Semidefinite programming for ad hoc wireless sensor network localization , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[28]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[29]  Michael Elad,et al.  Why Simple Shrinkage Is Still Relevant for Redundant Representations? , 2006, IEEE Transactions on Information Theory.

[30]  Don H. Johnson,et al.  To cooperate or not to cooperate: detection strategies in sensor networks , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[31]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[32]  Gabriel Cristóbal,et al.  Sparse overcomplete Gabor wavelet representation based on local competitions , 2006, IEEE Transactions on Image Processing.

[33]  I. Daubechies,et al.  PAINLESS NONORTHOGONAL EXPANSIONS , 1986 .

[34]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.

[35]  P. Casazza,et al.  Frames of subspaces , 2003, math/0311384.

[36]  A. Bacciotti,et al.  Liapunov functions and stability in control theory , 2001 .

[37]  Jean-Michel Poggi,et al.  Wavelets and their applications , 2007 .

[38]  Ronald A. DeVore,et al.  Image compression through wavelet transform coding , 1992, IEEE Trans. Inf. Theory.

[39]  Mark Coates,et al.  Evaluating causal relationships in wireless sensor/actuator networks , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[40]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[41]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[42]  Bruno A. Olshausen,et al.  Neural architectures for sparse approximation , 2008 .

[43]  S. Mallat A wavelet tour of signal processing , 1998 .

[44]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[45]  Robert D. Nowak,et al.  Distributed EM algorithms for density estimation and clustering in sensor networks , 2003, IEEE Trans. Signal Process..

[46]  Clarence Zener,et al.  Geometric Programming : Theory and Application , 1967 .

[47]  R.Balan,et al.  Density, Overcompleteness, and Localization of Frames. I. Theory , 2005, math/0510360.

[48]  Don H. Johnson,et al.  Point process models of single-neuron discharges , 1996, Journal of Computational Neuroscience.

[49]  R. Courant Methods of mathematical physics, Volume I , 1965 .

[50]  K. Cameron,et al.  Soil Science: Sustainable Production and Environmental Protection , 1996 .

[51]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[52]  J. Kovacevic,et al.  Life Beyond Bases: The Advent of Frames (Part II) , 2007, IEEE Signal Processing Magazine.

[53]  Wolfgang Porod,et al.  Qualitative analysis and synthesis of a class of neural networks , 1988 .

[54]  Ian F. Akyildiz,et al.  Wireless underground sensor networks : Research challenges , 2006 .

[55]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[57]  Maja J. Matarić,et al.  A market-based formulation of sensor-actuator network coordination , 2002, AAAI 2002.

[58]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[59]  R. Balan,et al.  Density, overcompleteness, and localization of frames , 2006 .

[60]  Richard G. Baraniuk,et al.  Neurally plausible sparse coding via thresholding and local competition , 2007 .

[61]  Gene F. Franklin,et al.  Feedback Control of Dynamic Systems , 1986 .

[62]  Don H. Johnson,et al.  Evaluating Local Contributions to Global Performance in Wireless Sensor and Actuator Networks , 2006, DCOSS.

[63]  V. Paulsen,et al.  Optimal frames for erasures , 2004 .

[64]  Dr. M. G. Worster Methods of Mathematical Physics , 1947, Nature.

[65]  M. Vetterli,et al.  Discrete directional wavelet bases and frames for image compression and denoising , 2003 .

[66]  Pascal Frossard,et al.  Flexible motion-adaptive video coding with redundant expansions , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[67]  R.L. Moses,et al.  Locating the nodes: cooperative localization in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[68]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[69]  E. Russell,et al.  Soil Physics , 1941, Nature.

[70]  D. Donoho Wedgelets: nearly minimax estimation of edges , 1999 .

[71]  D. Hubel Eye, brain, and vision , 1988 .

[72]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions with matching pursuit , 1994, Defense, Security, and Sensing.

[73]  N. Kingsbury Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[74]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[75]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[76]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[77]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[78]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[79]  Don H. Johnson,et al.  Optimizing physical layer data transmission for minimal signal distortion , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[80]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[81]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[82]  Simon J. Thorpe,et al.  Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit , 2004, Neurocomputing.

[83]  Alfred O. Hero,et al.  Distributed maximum likelihood estimation for sensor networks , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[84]  Don H. Johnson,et al.  Reconstruction of compressively sensed images via neurally plausible local competitive algorithms , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[85]  Andrea J. Goldsmith,et al.  Joint modulation and multiple access optimization under energy constraints , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[86]  R. Decarlo,et al.  Perspectives and results on the stability and stabilizability of hybrid systems , 2000, Proceedings of the IEEE.

[87]  Raymon M. Glantz,et al.  The crayfish sustaining fibers , 1983, Journal of comparative physiology.

[88]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[89]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[90]  Avideh Zakhor,et al.  Video compression using matching pursuits , 1999, IEEE Trans. Circuits Syst. Video Technol..

[91]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[92]  R M Glantz,et al.  Linear integration of convergent visual inputs in an oculomotor reflex pathway. , 1984, Journal of neurophysiology.

[93]  M. Picheny,et al.  Towards super-human speech recogniton , 2003 .

[94]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[95]  Ronald R. Coifman,et al.  In Wavelets and Statistics , 1995 .

[96]  Justin K. Romberg,et al.  Wavelet-domain approximation and compression of piecewise smooth images , 2006, IEEE Transactions on Image Processing.

[97]  O. Christensen An introduction to frames and Riesz bases , 2002 .

[98]  P. Casazza,et al.  Fusion frames and distributed processing , 2006, math/0605374.

[99]  Russell Fish,et al.  Keep your eye on the ball , 1989, SIFN.

[100]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition , 2006 .

[101]  Don H. Johnson,et al.  Analysis of noise reduction in redundant expansions under distributed processing requirements , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[102]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[103]  Christopher J. Rozell,et al.  Analyzing dynamics and stimulus feature dependence in the information processing of crayfish sustaining fibers , 2002 .

[104]  B. Delgutte,et al.  Neural coding of the temporal envelope of speech : Relation to modulation transfer functions , 2001 .

[105]  Robert W. Heath,et al.  Designing structured tight frames via an alternating projection method , 2005, IEEE Transactions on Information Theory.

[106]  Millard F. Reschke,et al.  Compensatory Eye Movements , 2008 .

[107]  Andrea J. Goldsmith,et al.  Energy-efficiency of MIMO and cooperative MIMO techniques in sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[108]  J. Gallant,et al.  Natural Stimulation of the Nonclassical Receptive Field Increases Information Transmission Efficiency in V1 , 2002, The Journal of Neuroscience.

[109]  J. Rothwell Principles of Neural Science , 1982 .

[110]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[111]  Jelena Kovačević,et al.  Life Beyond Bases: The Advent of Frames , 2006 .

[112]  Bruno A. Olshausen,et al.  Principles of Image Representation in Visual Cortex , 2003 .

[113]  Leonidas J. Guibas,et al.  Wireless sensor networks - an information processing approach , 2004, The Morgan Kaufmann series in networking.

[114]  Feng Zhao,et al.  State-Centric Programming for Sensor-Actuator Network Systems , 2003, IEEE Pervasive Comput..

[115]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[116]  John G. Proakis,et al.  Digital Communications , 1983 .

[117]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[118]  Junbin Gao,et al.  On a Class of Support Vector Kernels Based on Frames in Function Hilbert Spaces , 2001, Neural Computation.

[119]  T. Yamaguchi,et al.  The neuronal components of the optic nerve of the crayfish as studied by single unit analysis , 1966, The Journal of comparative neurology.

[120]  Robert D. Nowak,et al.  Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical imaging , 2003, IEEE Transactions on Medical Imaging.

[121]  Emmanuel J. Candès,et al.  Error correction via linear programming , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[122]  D. Vere-Jones Markov Chains , 1972, Nature.

[123]  B. R. Johnson,et al.  All-optical nanoscale pH meter. , 2006, Nano letters.

[124]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[125]  P. O. Bishop,et al.  Spatial vision. , 1971, Annual review of psychology.

[126]  Richard G. Baraniuk,et al.  Neurally plausible sparse coding via competitive algorithms , 2007 .

[127]  Jonathan Waxman,et al.  Information Theory and Neuroscience , 2009 .

[128]  H. Vincent Poor,et al.  Collaborative beamforming for distributed wireless ad hoc sensor networks , 2005, IEEE Transactions on Signal Processing.

[129]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[130]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[131]  A. Morse,et al.  Stability of switched systems with average dwell-time , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[132]  Nick G. Kingsbury,et al.  Redundant representation with complex wavelets: how to achieve sparsity , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[133]  Alan V. Oppenheim,et al.  Quantization Noise Shaping on Arbitrary Frame Expansions , 2006, EURASIP J. Adv. Signal Process..

[134]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[135]  Andrea J. Goldsmith,et al.  Power scheduling of universal decentralized estimation in sensor networks , 2006, IEEE Transactions on Signal Processing.

[136]  Andreas Turk,et al.  Hierarchical parallel matching pursuit , 1994, Optics & Photonics.

[137]  Christopher J. Rozell,et al.  Matched filter performance for unequal target and background covariance matrices , 2004, SPIE Defense + Commercial Sensing.

[138]  Kenneth Zeger,et al.  Quantization of Multiple Sources Using Nonnegative Integer Bit Allocation , 2006, IEEE Transactions on Information Theory.

[139]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[140]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[141]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[142]  Nicolai Petkov,et al.  Fast atomic decomposition by the inhibition method , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[143]  Jian Fan,et al.  Frame representations for texture segmentation , 1996, IEEE Trans. Image Process..

[144]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[145]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[146]  Pascal Frossard,et al.  A posteriori quantization of progressive matching pursuit streams , 2004, IEEE Transactions on Signal Processing.

[147]  Urbashi Mitra,et al.  Estimating inhomogeneous fields using wireless sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[148]  Kannan Ramchandran,et al.  Denoising by Sparse Approximation: Error Bounds Based on Rate-Distortion Theory , 2006, EURASIP J. Adv. Signal Process..

[149]  John W. Fisher,et al.  Nonparametric belief propagation for self-localization of sensor networks , 2005, IEEE Journal on Selected Areas in Communications.

[150]  Stephen J. Wright Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[151]  Robert W. Heath,et al.  Linear dispersion codes for MIMO systems based on frame theory , 2002, IEEE Trans. Signal Process..

[152]  Tharam S. Dillon,et al.  Exponential stability and oscillation of Hopfield graded response neural network , 1994, IEEE Trans. Neural Networks.

[153]  Christopher Heil,et al.  Continuous and Discrete Wavelet Transforms , 1989, SIAM Rev..

[154]  John J. Benedetto,et al.  Sigma-delta quantization and finite frames , 2004, ICASSP.

[155]  R. A. Davidoff From Neuron to Brain , 1977, Neurology.

[156]  O. Norwood Density , 1993, International Society of Hair Restoration Surgery.

[157]  L. Rebollo-Neira,et al.  Optimized orthogonal matching pursuit approach , 2002, IEEE Signal Processing Letters.

[158]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[159]  Don H. Johnson,et al.  Analyzing the robustness of redundant population codes in sensory and feature extraction systems , 2006, Neurocomputing.

[160]  R. Duffin,et al.  A class of nonharmonic Fourier series , 1952 .

[161]  Gregory J. Pottie,et al.  Instrumenting the world with wireless sensor networks , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[162]  A. Forge Psychophysical and physiological advances in hearing , 1998 .

[163]  Christopher J. Rozell,et al.  Distributed processing in frames for sparse approximation , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[164]  Avideh Zakhor,et al.  Very low bit-rate video coding based on matching pursuits , 1997, IEEE Trans. Circuits Syst. Video Technol..

[165]  Shayne Waldron,et al.  Generalized Welch bound equality sequences are tight fram , 2003, IEEE Trans. Inf. Theory.

[166]  Don H. Johnson,et al.  Feature-Based Information Processing with Selective Attention , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[167]  Michael S. Lewicki,et al.  Efficient coding of natural sounds , 2002, Nature Neuroscience.

[168]  Simon J. Thorpe,et al.  Coding static natural images using spiking event times: do neurons Cooperate? , 2004, IEEE Transactions on Neural Networks.

[169]  Richard G. Baraniuk,et al.  The embedded triangles algorithm for distributed estimation in sensor networks , 2003, IEEE Workshop on Statistical Signal Processing, 2003.