Information Theory and Coding by Example

This fundamental monograph introduces both the probabilistic and algebraic aspects of information theory and coding. It has evolved from the authors' years of experience teaching at the undergraduate level, including several Cambridge Maths Tripos courses. The book provides relevant background material, a wide range of worked examples and clear solutions to problems from real exam papers. It is a valuable teaching aid for undergraduate and graduate students, or for researchers and engineers who want to grasp the basic principles.

[1]  Thomas M. Thompson.  From error-correcting codes through sphere packings to simple groups , 1983 .

[2]  Darrel Hankerson,et al.  Introduction to Information Theory and Data Compression , 2003 .

[3]  Branka Vucetic,et al.  Turbo Codes: Principles and Applications , 2000 .

[4]  A. Wyner.  A note on the capacity of the band-limited Gaussian channel , 1966, The Bell System Technical Journal.

[5]  Judy L. Walker,et al.  Codes and Curves , 2000, Student mathematical library.

[6]  V. D. Goppa.  Geometry and Codes , 1988 .

[7]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[8]  Mark Kelbert,et al.  Un upper bound for Kullback-Leibler divergence with a small number of outliers , 2013 .

[9]  R. Mollin.  RSA and Public-Key Cryptography , 2002 .

[10]  P. Shields.  The Ergodic Theory of Discrete Sample Paths , 1996 .

[11]  Ian F. Blake,et al.  The mathematical theory of coding , 1975 .

[12]  W. Ebeling,et al.  Lattices and Codes: A Course Partially Based on Lectures by F. Hirzebruch , 1994 .

[13]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[14]  Torleiv Kløve,et al.  Codes for Error Detection , 2007, Series on Coding Theory and Cryptology.

[15]  Mark Kelbert,et al.  Continuity of mutual entropy in the large signal-to-noise ratio limit , 2009, ArXiv.

[16]  M. Shrikhande,et al.  Quasi-Symmetric Designs , 1991 .

[17]  Daniel Augot,et al.  A Note on the Generalisation of the Guruswami-Sudan List Decoding Algorithm to Reed-Muller Codes , 2009, Gröbner Bases, Coding, and Cryptography.

[18]  Gregory J. Chaitin,et al.  Information-Theoretic Incompleteness , 1992, World Scientific Series in Computer Science.

[19]  S. G. Vladut,et al.  Algebraic-Geometric Codes , 1991 .

[20]  Douglas R. Stinson,et al.  Cryptography: Theory and Practice , 1995 .

[21]  Robert U. Ayres,et al.  Manufacturing and Human Labor as Information Processes , 1987 .

[22]  Aiden A. Bruen,et al.  Cryptography, information theory, and error-correction - a handbook for the 21st century , 2005, Wiley-Interscience series in discrete mathematics and optimization.

[23]  S. Engelberg.  Random Signals and Noise , 2006 .

[24]  Salvatore Gravano,et al.  Introduction to Error Control Codes , 2001 .

[25]  David Chapman,et al.  Random signals and noise , 1992 .

[26]  M. J. Usher,et al.  Information and Communication for Engineers , 1997 .

[27]  François Baccelli,et al.  A Palm theory approach to error exponents , 2008, 2008 IEEE International Symposium on Information Theory.

[28]  P. Farrell,et al.  Essentials of Error-control Coding , 2006 .

[29]  W. W. Peterson,et al.  Error-Correcting Codes. , 1962 .

[30]  W. E. Hartnett.  Foundations of Coding Theory , 1974 .

[31]  J. H. Lint,et al.  Introduction to coding theory and algebraic geometry , 1989 .

[32]  Shu Lin,et al.  An introduction to error-correcting codes , 1970 .

[33]  Peter Seibt.  Algorithmic Information Theory: Mathematics of Digital Information Processing , 2010 .

[34]  R. Michael Buehrer,et al.  Impact of exclusion region and spreading in spectrum-sharing ad hoc networks , 2006, TAPAS '06.

[35]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[36]  J. Berstel,et al.  Theory of codes , 1985 .

[37]  Christian Schlegel,et al.  Trellis and turbo coding , 2004 .

[38]  Franklin M. Ingels.  Information and coding theory , 1971 .

[39]  N. Koblitz.  A Course in Number Theory and Cryptography , 1987 .

[40]  D. Stoyan,et al.  Stochastic Geometry and Its Applications , 1989 .

[41]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[42]  I. M. Yaglom,et al.  Probability and Information , 1984 .

[43]  W. Cary Huffman,et al.  Fundamentals of Error-Correcting Codes , 1975 .

[44]  Steven Roman,et al.  Coding and information theory , 1992 .

[45]  Johannes A. Buchmann,et al.  Introduction to Cryptography , 2001 .

[46]  A. Myasnikov,et al.  Group-based Cryptography , 2008 .

[47]  Flo Conway,et al.  Dark Hero of the Information Age: In Search of Norbert Wiener, The Father of Cybernetics , 2006 .

[48]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[49]  R. Morelos-Zaragoza.  The art of error correcting coding , 2002 .

[50]  Juergen Bierbrauer.  Introduction to coding theory , 2005, Discrete mathematics and its applications.

[51]  E. Lieb.  Proof of an entropy conjecture of Wehrl , 1978 .

[52]  Jaroslava Mikulecká.  On a hybrid experimental design , 1983, Kybernetika.

[53]  David Applebaum,et al.  Probability and Information: An Integrated Approach , 2008 .

[54]  Peter Grassberger,et al.  Entropy estimation of symbol sequences. , 1996, Chaos.

[55]  I. Blake,et al.  An Introduction to Algebraic and Combinatorial Coding Theory , 1976 .

[56]  Abraham Sinkov,et al.  Elementary Cryptanalysis: A Mathematical Approach , 1970 .

[57]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[58]  O. Johnson.  Information Theory And The Central Limit Theorem , 2004 .

[59]  V. Pless.  Introduction to the Theory of Error-Correcting Codes , 1991 .

[60]  David G. Luenberger,et al.  Information Science , 2006 .

[61]  Peter J. Cameron,et al.  Designs, graphs, codes, and their links , 1991 .

[62]  Thammavarapu R. N. Rao,et al.  Error coding for arithmetic processors , 1974 .

[63]  Stanford Goldman,et al.  Information theory , 1953 .

[64]  Aleksandr Yakovlevich Khinchin,et al.  Mathematical foundations of information theory , 1959 .

[65]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[66]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[67]  Gregory J. Chaitin,et al.  Algorithmic Information Theory , 1987, IBM J. Res. Dev..

[68]  C. Helstrom,et al.  Statistical theory of signal detection , 1968 .

[69]  Jacob Wolfowitz.  Coding Theorems of Information Theory , 1962 .

[70]  R. Lidl,et al.  Applied abstract algebra , 1984 .

[71]  Jacobus H. van Lint,et al.  Coding Theory , 1971 .

[72]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[73]  J. A. Llewellyn.  Information and Coding , 1987 .

[74]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[75]  Noam D. Elkies,et al.  Excellent codes from modular curves , 2001, STOC '01.

[76]  R. Lewand.  Cryptological Mathematics , 2000 .

[77]  Robert J. McEliece,et al.  The Theory of Information and Coding , 1979 .

[78]  Michel Loève,et al.  Probability Theory I , 1977 .

[79]  V. Maslov,et al.  On Minimization and Maximization of Entropy in Various Disciplines , 2004 .

[80]  Carl W. Helstrom,et al.  Elements of signal detection and estimation , 1994 .

[81]  Elwyn R. Berlekamp,et al.  A survey of algebraic coding theory , 1970 .

[82]  Aaron D. Wyner.  Capacity of the Product of Channels , 1966, Inf. Control..

[83]  Richard E. Blahut,et al.  Algebraic codes on lines, planes, and curves , 2008 .

[84]  M. J. Usher.  Information Theory for Information Technologists , 1984 .

[85]  Steve J. Heims,et al.  John Von Neumann and Norbert Wiener, From Mathematics to the Technologies of Life and Death , 1980 .

[86]  William Stallings,et al.  Cryptography and Network Security: Principles and Practice , 1998 .

[87]  J. Baylis.  Error-correcting Codes , 2014 .

[88]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[89]  William G. Chambers.  Basics of Communications and Coding , 1985 .

[90]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[91]  Rudolf Lide,et al.  Finite fields , 1983 .

[92]  Tracey Ho,et al.  Network Coding: An Introduction , 2008 .

[93]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[94]  A. Kerber,et al.  Error-correcting linear codes : classification by isometry and applications , 2006 .

[95]  Jørn Justesen,et al.  Class of constructive asymptotically good algebraic codes , 1972, IEEE Trans. Inf. Theory.

[96]  T. Han,et al.  Mathematics of information and coding , 2001 .

[97]  R. Blahut.  Theory and practice of error control codes , 1983 .

[98]  Ron M. Roth,et al.  Introduction to Coding Theory , 2019, Discrete Mathematics.

[99]  H. Niederreiter,et al.  Rational Points on Curves Over Finite Fields: Theory and Applications , 2001 .

[100]  Gabriele Nebe,et al.  Self-dual codes and invariant theory , 2009, Algebraic Aspects of Digital Communications.

[101]  Philippe Piret,et al.  Convolutional Codes: An Algebraic Approach , 1988 .

[102]  Mike Prest,et al.  Numbers, Groups and Codes , 1989 .

[103]  Andrei Fionov,et al.  Basics of Contemporary Cryptography for IT Practitioners , 2005, Series on Coding Theory and Cryptology.

[104]  D. Slepian,et al.  Prolate spheroidal wave functions, fourier analysis and uncertainty — II , 1961 .

[105]  Oliver Pretzel.  Error-Correcting Codes and Finite Fields , 1992 .

[106]  H. Pollak,et al.  Prolate spheroidal wave functions, fourier analysis and uncertainty — III: The dimension of the space of essentially time- and band-limited signals , 1962 .

[107]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[108]  Henning Stichtenoth,et al.  Algebraic function fields and codes , 1993, Universitext.

[109]  Graham Wade.  Coding Techniques: An Introduction to Compression and Error Control , 2000 .

[110]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[111]  Jirí Adámek.  Foundations of coding - theory and applications of error-correcting codes with an introduction to cryptography and information theory , 1991 .

[112]  Jennifer D. Key,et al.  Designs and their codes , 1992, Cambridge tracts in mathematics.

[113]  G. Jones,et al.  Information and Coding Theory , 2000 .

[114]  R. Gray.  Source Coding Theory , 1989 .

[115]  Raymond W. Yeung,et al.  A First Course in Information Theory , 2002 .

[116]  Wade Trappe,et al.  Introduction to Cryptography with Coding Theory , 2002 .

[117]  Shu Lin,et al.  Channel Codes: Classical and Modern , 2009 .

[118]  C.J.S. deSilva,et al.  Fundamentals of Information Theory and Coding Design , 2003 .

[119]  Fazlollah M. Reza,et al.  Introduction to Information Theory , 2004, Lecture Notes in Electrical Engineering.

[120]  Venkatesan Guruswami,et al.  Improved decoding of Reed-Solomon and algebraic-geometry codes , 1999, IEEE Trans. Inf. Theory.

[121]  M. Tsfasman,et al.  Modular curves, Shimura curves, and Goppa codes, better than Varshamov‐Gilbert bound , 1982 .

[122]  Elwyn R. Berlekamp,et al.  Algebraic coding theory , 1984, McGraw-Hill series in systems science.

[123]  S. Kullback,et al.  Topics in statistical information theory , 1987 .

[124]  Amiel Feinstein,et al.  Foundations of Information Theory , 1959 .

[125]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[126]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[127]  Nelson M. Blachman,et al.  Noise and its effect on Communication , 1966 .

[128]  P. Jones,et al.  A Diary on Information Theory , 1989 .

[129]  Gregory J. Chaitin.  The LIMITS of MATHEMATICS: A Course on Information Theory and the Limits of Formal Reasoning , 2012 .

[130]  F. MacWilliams,et al.  The Theory of Error-Correcting Codes , 1977 .

[131]  Ray Hill,et al.  A First Course in Coding Theory , 1988 .

[132]  Gary L. Mullen,et al.  Finite Fields and Applications , 2007, Student mathematical library.

[133]  Ioan Mackenzie James.  Remarkable Mathematicians: From Euler to von Neumann , 2002 .

[134]  Kevin T. Phelps,et al.  Coding Theory and Cryptography : The Essentials , 2022 .

[135]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .

[136]  D. S. Jones,et al.  Elementary information theory , 1979 .

[138]  P. Billingsley,et al.  Ergodic theory and information , 1966 .

[139]  Freeman J. Dyson,et al.  The Tragic Tale of a Genius , 2005 .

[140]  I. Vajda.  Theory of statistical inference and information , 1989 .

[141]  Lekh R. Vermani.  Elements of Algebraic Coding Theory , 1996 .

[142]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[143]  Slavko Simic.  Best possible global bounds for Jensen functional , 2010 .

[144]  Dominic J. A. Welsh,et al.  Codes and cryptography , 1988 .

[145]  Sergio Verdú,et al.  A simple proof of the entropy-power inequality , 2006, IEEE Transactions on Information Theory.

[146]  Richard W. Hamming,et al.  Coding and Information Theory , 1980 .

[147]  Peter F. Sweeney.  Error control coding - an introduction , 1991 .

[148]  M. Tsfasman,et al.  Algebraic Geometric Codes: Basic Notions , 2007 .