Annealing Based Optimization Methods for Signal Processing Applications

In this thesis, a class of combinatorial optimization methods rooted in statistical mechanics and their use in signal processing applications will be discussed. The thesis consists of two separate parts. The first part deals with the rationale for my work and also covers the background information necessary to put the second part, which consists of a number of papers, in context. There are (at least) two sides to an optimization problem---the problem statement arising from an application or a design and the selection of an algorithm to solve the problem. In this work the problem statements are practical problems, of combinatorial nature, frequently encountered in signal processing and the algorithms of choice are annealing based algorithms, founded in statistical mechanics. From my work, it is my experience that solving a particular problem often leads to new developments on the part of the algorithm which, in turn, open up possibilities to apply the modified algorithm to a new set of problems, leading to a continuously improving algorithm and a growing field of applications. The included papers deal with the application of annealing optimization methods to the problems of configuring active noise and vibration control systems, digital filter design and adaptive filtering. They also describe the successive development of a highly efficient entropy-directed deterministic annealing (EDDA) optimization algorithm detailed in the final paper.

[1]  Earl E. Swartzlander,et al.  Computer Arithmetic , 1980 .

[2]  Sathyanarayan S. Rao,et al.  Design of discrete coefficient FIR filters by simulated evolution , 1996, IEEE Signal Processing Letters.

[3]  A. Roure,et al.  ACTIVE NOISE CONTROL OF ACOUSTIC SOURCES USING SPHERICAL HARMONICS EXPANSION AND A GENETIC ALGORITHM: SIMULATION AND EXPERIMENT , 1998 .

[4]  Fred Glover,et al.  Tabu Search: A Tutorial , 1990 .

[5]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Stephen J. Wright,et al.  Interior-point methods , 2000 .

[7]  Shen Lin.  Computer solutions of the traveling salesman problem , 1965 .

[8]  Yong Ching Lim,et al.  Signed power-of-two term allocation scheme for the design of digital filters , 1999 .

[9]  B. Widrow,et al.  The complex LMS algorithm , 1975, Proceedings of the IEEE.

[10]  Carsten Peterson,et al.  An efficient mean field approach to the set covering problem , 1999, Eur. J. Oper. Res..

[11]  Eugene L. Lawler,et al.  Traveling Salesman Problem , 2016 .

[12]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[13]  M. Kalkan,et al.  Design and discrete re-optimisation of all-pass based power symmetric IIR filters , 1996 .

[14]  Thomas L Lagö,et al.  Active Vibration Control on a Light High Speed Train : Preliminary Results , 1997 .

[15]  Sen M. Kuo,et al.  Integrated automotive signal processing and audio system , 1993 .

[16]  Keh-Shew Lu,et al.  DIGITAL FILTER DESIGN , 1973 .

[17]  S. C. Southward,et al.  Active Control of Noise and Vibration , 1996 .

[18]  Lars Håkansson,et al.  Active Control of Lateral Vibrations in a Railway Carriage , 2002 .

[19]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..

[20]  J. Burgess.  Active adaptive sound control in a duct: A computer simulation , 1981 .

[21]  Ingvar Claesson,et al.  Evaluation of the Performance of an Active Noise Control System in an Aircraft Mock-up , 2000 .

[22]  Sanjit K. Mitra,et al.  A new approach to the realization of low-sensitivity IIR digital filters , 1986, IEEE Trans. Acoust. Speech Signal Process..

[23]  Philip A. Nelson,et al.  The active minimization of harmonic enclosed sound fields, part II: A computer simulation , 1987 .

[24]  Tevfik Bultan,et al.  A fast neural-network algorithm for VLSI cell placement , 1998, Neural Networks.

[25]  Ingvar Claesson,et al.  Considerations on Large Applications of Active Noise Control, Part I : Theory , 2003 .

[26]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[27]  Mathias Winberg,et al.  AVIIS, ACTIVE VIBRATION ISOLATION IN SHIPS : SOURCE IDENTIFICATION AND ANVC APPROACH , 1997 .

[28]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[29]  Ibrahim M. Elfadel,et al.  Convex Potentials and their Conjugates in Analog Mean-Field Optimization , 1995, Neural Computation.

[30]  Ingvar Claesson,et al.  Comparison of Multiple- and Single-Reference MIMO Active Noise Control Approaches Using Data Measured in a Dornier 328 Aircraft , 2000 .

[31]  Ingvar Claesson,et al.  Design of digital filters with general hardware constraints by mean field annealing , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[32]  Ingvar Claesson,et al.  Convergence analysis of a twin-reference complex least-mean-squares algorithm , 2002, IEEE Trans. Speech Audio Process..

[33]  S. Nordebo,et al.  Hardware efficient digital filter design by multimode mean field annealing , 2001, IEEE Signal Processing Letters.

[34]  Lars Håkansson,et al.  Feedback-LMS control of lateral vibration in a train car , 2001 .

[35]  Patrice Koehl,et al.  Building protein lattice models using self-consistent mean field theory , 1998 .

[36]  Robert E. Beck,et al.  Elementary Linear Programming with Applications , 1979 .

[37]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[38]  Gerhard Schmidt,et al.  Acoustic echo control. An application of very-high-order adaptive filters , 1999, IEEE Signal Process. Mag..

[39]  Johnny Holmberg.  On Design, Analysis, and Implementation of LDI/LDD Lattice Filters , 2002 .

[40]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[41]  Zhi-Quan Luo,et al.  Adaptive linear filtering using interior point optimization techniques , 2000, IEEE Trans. Signal Process..

[42]  Lars Wanhammar.  DSP integrated circuits , 1999 .

[43]  Ingvar Claesson,et al.  Multireference controllers for active control of noise and vibration , 1996 .

[44]  Carlos A. Coello Coello,et al.  THEORETICAL AND NUMERICAL CONSTRAINT-HANDLING TECHNIQUES USED WITH EVOLUTIONARY ALGORITHMS: A SURVEY OF THE STATE OF THE ART , 2002 .

[45]  Mattias Dahl,et al.  Complex Approximation by Semi-Infinite Quadratic Programming , 2000 .

[46]  Yong Lian,et al.  A polynomial-time algorithm for designing FIR filters with power-of-two coefficients , 2002, IEEE Trans. Signal Process..

[47]  Anthony J. G. Hey,et al.  Feynman Lectures on Computation , 1996 .

[48]  Tapio Saramäki,et al.  An efficient algorithm for the design of lattice wave digital filters with short coefficient wordlength , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[49]  E. Dubois,et al.  Design of multidimensional finite-wordlength FIR and IIR filters by simulated annealing , 1995 .

[50]  Hai Huyen Dam,et al.  Approximation of classical IIR filters with additional specifications , 2000 .

[51]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[52]  Andreas Antoniou,et al.  Digital Filters: Analysis, Design and Applications , 1979 .

[53]  Alan V. Oppenheim,et al.  Discrete-Time Signal Pro-cessing , 1989 .

[54]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[55]  李幼升,et al.  Ph , 1989 .

[56]  Ingvar Claesson,et al.  AVIIS, ACTIVE VIBRATION ISOLATION IN SHIPS; AN ASAC APPROACH , 1999 .

[57]  Lars Håkansson,et al.  Active Control Methods for Reducing Lateral Vibrations in High-Speed Trains , 2002 .

[58]  Emile H. L. Aarts,et al.  A new polynomial time cooling schedule , 1985 .

[59]  鈴木 増雄.  Time-Dependent Statistics of the Ising Model , 1965 .

[60]  Ehl Emile Aarts,et al.  Simulated annealing and Boltzmann machines , 2003 .

[61]  Ingvar Claesson,et al.  ACTIVE CONTROL OF PROPELLER-INDUCED NOISE IN AN AIRCRAFT MOCK-UP , 1999 .

[62]  G. P. Eatwell,et al.  Active control of vibration , 1992 .

[63]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[64]  M. Waterman,et al.  Mapping DNA by stochastic relaxation , 1987 .

[65]  B. Widrow,et al.  Adaptive inverse control , 1987, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[66]  L. E. Turner,et al.  Design of digital filters using simulated annealing , 1990, IEEE International Symposium on Circuits and Systems.

[67]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[68]  M. O. Tokhi,et al.  Active noise control systems , 1987 .

[69]  A. Fettweis.  Wave digital filters: Theory and practice , 1986, Proceedings of the IEEE.

[70]  J. Shynk.  Frequency-domain and multirate adaptive filtering , 1992, IEEE Signal Processing Magazine.

[71]  Stephen J. Elliott,et al.  Natural algorithms for choosing source locations in active control systems , 1995 .

[72]  Per Eriksson,et al.  Efficient filter structures for active noise, LMS controllers , 1991 .

[73]  Ingvar Claesson,et al.  An adaptive filtering algorithm using mean field annealing techniques , 2002, IEEE Workshop on Signal Processing Systems.

[74]  Carsten Peterson,et al.  A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[75]  G. A. Jordan,et al.  Forest landscape management modeling using simulated annealing , 2002 .

[76]  H. Szu.  Fast simulated annealing , 1987 .

[77]  Tapio Saramäki,et al.  An algorithm for the design of multiplierless approximately linear-phase lattice-wave digital filters , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[78]  George V. Moustakides.  Study of the transient phase of the forgetting factor RLS , 1997, IEEE Trans. Signal Process..

[79]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[80]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[81]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[82]  Albert Y. Zomaya,et al.  Multiprocessor scheduling using mean-field annealing , 1998, Future Gener. Comput. Syst..

[83]  Monson H. Hayes,et al.  Statistical Digital Signal Processing and Modeling , 1996 .

[84]  Carsten Peterson,et al.  Neural Networks for Optimization Problems with Inequality Constraints: The Knapsack Problem , 1993, Neural Computation.

[85]  Per Persson,et al.  Active vibration reduction in a light weight high speed train bogie , 1999 .

[86]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[87]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[88]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[89]  Ingvar Claesson,et al.  Considerations on Large Applications of Active Noise Control, Part II : Experimental Results , 2003 .

[90]  Colin H. Hansen,et al.  Active control of vibration , 1996 .

[91]  Jong-Jy Shyu,et al.  A new approach to the design of discrete coefficient FIR digital filters , 1995, IEEE Trans. Signal Process..

[92]  A. N. Willson,et al.  Insights into digital filters made as the sum of two allpass functions , 1995 .

[93]  Michele Marchesi,et al.  Applications of simulated annealing for the design of special digital filters , 1992, IEEE Trans. Signal Process..

[94]  D. F. Wong,et al.  Simulated Annealing for VLSI Design , 1988 .

[95]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[96]  Philip A. Nelson,et al.  The active minimization of harmonic enclosed sound fields, part I: Theory , 1987 .

[97]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[98]  J. J. Kosowsky,et al.  Statistical Physics Algorithms That Converge , 1994, Neural Computation.

[99]  Ingvar Claesson,et al.  A Novel Multiple-Reference Algorithm for Active Control of Propeller-Induced Noise in Aircraft Cabins , 2000 .

[100]  Kurt Binder,et al.  A Guide to Monte Carlo Simulations in Statistical Physics , 2000 .

[101]  Lars Håkansson,et al.  Adaptive Active Control of Machine-Tool Vibration in a Lathe - Analysis and Experiments , 1998 .

[102]  Geoffrey C. Fox,et al.  Vector quantization by deterministic annealing , 1992, IEEE Trans. Inf. Theory.

[103]  Ingvar Claesson,et al.  Active Noise Control in a Twin-Engine Patrol Boat , 1996 .

[104]  Ingvar Claesson,et al.  Evaluation of multiple reference active noise control algorithms on Dornier 328 aircraft data , 1999, IEEE Trans. Speech Audio Process..

[105]  Holger Blume,et al.  FIR-filter design with spatial and frequency design constraints using evolution strategies , 1998, Signal Process..

[106]  Per Persson,et al.  Active Control of Sleeper-induced Sound in a High Speed Train , 1999 .

[107]  Dirk T. M. Slock,et al.  On the convergence behavior of the LMS and the normalized LMS algorithms , 1993, IEEE Trans. Signal Process..