An Overview of Capacity Results for Synchronization Channels

Synchronization channels, such as the well-known deletion channel, are surprisingly harder to analyze than memoryless channels, and they are a source of many fundamental problems in information theory and theoretical computer science. One of the most basic open problems regarding synchronization channels is the derivation of an exact expression for their capacity. Unfortunately, most of the classic information-theoretic techniques at our disposal fail spectacularly when applied to synchronization channels. Therefore, new approaches must be considered to tackle this problem. This survey gives an account of the great effort made over the past few decades to better understand the (broadly defined) capacity of synchronization channels, including both the main results and the novel techniques underlying them. Besides the usual notion of channel capacity, we also discuss the zero-error capacity of synchronization channels.

[1]  Vahid Tarokh,et al.  A survey of error-correcting codes for channels with symbol synchronization errors , 2010, IEEE Communications Surveys & Tutorials.

[2]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[3]  David Zuckerman,et al.  Asymptotically good codes correcting insertions, deletions, and transpositions , 1997, SODA '97.

[4]  Jie Ma,et al.  Longest Common Subsequences in Sets of Words , 2014, SIAM J. Discret. Math..

[5]  Tolga M. Duman,et al.  Upper Bounds on the Capacity of Deletion Channels Using Channel Fragmentation , 2015, IEEE Transactions on Information Theory.

[6]  Dario Fertonani,et al.  Bounds on the Capacity of Channels with Insertions, Deletions and Substitutions , 2011, IEEE Transactions on Communications.

[7]  Majid Khabbazian,et al.  On the Number of Subsequences When Deleting Symbols From a String , 2008, IEEE Transactions on Information Theory.

[8]  Jirí Matousek,et al.  Expected Length of the Longest Common Subsequence for Large Alphabets , 2003, LATIN.

[9]  Bruce Spang,et al.  Coded trace reconstruction in a constant number of traces , 2020, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[10]  Yashodhan Kanoria,et al.  On the deletion channel with small deletion probability , 2009, 2010 IEEE International Symposium on Information Theory.

[11]  Rudolf Ahlswede,et al.  CHANNELS WITHOUT SYNCHRONIZATION , 1971 .

[12]  A. J. Han Vinck,et al.  A coding scheme for single peak-shift correction in (d, k)-constrained channels , 1993, IEEE Trans. Inf. Theory.

[13]  Alon Orlitsky,et al.  Interactive communication: balanced distributions, correlated files, and average-case complexity , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[14]  Sergio Verdú,et al.  On channel capacity per unit cost , 1990, IEEE Trans. Inf. Theory.

[15]  Mahdi Cheraghchi,et al.  Improved Upper Bounds and Structural Results on the Capacity of the Discrete-Time Poisson Channel , 2019, IEEE Transactions on Information Theory.

[16]  Vladimir I. Levenshtein.  Efficient Reconstruction of Sequences from Their Subsequences or Supersequences , 2001, J. Comb. Theory, Ser. A.

[17]  Bernhard Haeupler.  Optimal Document Exchange and New Codes for Insertions and Deletions , 2019, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).

[18]  Suhas N. Diggavi,et al.  On information transmission over a finite buffer channel , 2000, IEEE Transactions on Information Theory.

[19]  Mladen Kovačević.  Error-Free Communication in the Presence of Duplication Errors. , 2019 .

[20]  Wei Zeng,et al.  On the Information Stability of Channels With Timing Errors , 2006, 2006 IEEE International Symposium on Information Theory.

[21]  Jehoshua Bruck,et al.  Correcting Deletions in Multiple-Heads Racetrack Memories , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[22]  Eitan Yaakobi,et al.  Reconstruction from Deletions in Racetrack Memories , 2018, 2018 IEEE Information Theory Workshop (ITW).

[23]  Lara Dolecek,et al.  Coding for Deletion Channels with Multiple Traces , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[24]  Venkatesan Guruswami,et al.  Optimally resilient codes for list-decoding from insertions and deletions , 2019, Electron. Colloquium Comput. Complex..

[25]  S. Shamai,et al.  Capacity of a pulse amplitude modulated direct detection photon channel , 1990 .

[26]  N.J.A. Sloane,et al.  On Single-Deletion-Correcting Codes , 2002, math/0207197.

[27]  Eitan Yaakobi,et al.  Coding for racetrack memories , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[28]  Michael Mitzenmacher,et al.  A Simple Lower Bound for the Capacity of the Deletion Channel , 2006, IEEE Transactions on Information Theory.

[29]  Michael Mitzenmacher,et al.  Repeated deletion channels , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[30]  Patrick A. H. Bours.  Construction of fixed-length insertion/deletion correcting runlength-limited codes , 1994, IEEE Trans. Inf. Theory.

[31]  Adam Tauman Kalai,et al.  Tight asymptotic bounds for the deletion channel with small deletion probabilities , 2010, 2010 IEEE International Symposium on Information Theory.

[32]  Michael E. Saks,et al.  Approximating Edit Distance within Constant Factor in Truly Sub-Quadratic Time , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[33]  Olgica Milenkovic,et al.  Coded Trace Reconstruction , 2019, 2019 IEEE Information Theory Workshop (ITW).

[34]  Ibrahim C. Abou-Faycal,et al.  The capacity of discrete-time memoryless Rayleigh-fading channels , 2001, IEEE Trans. Inf. Theory.

[35]  Yashodhan Kanoria,et al.  Optimal Coding for the Binary Deletion Channel With Small Deletion Probability , 2013, IEEE Transactions on Information Theory.

[36]  G. Doerk,et al.  Bit-Patterned Magnetic Recording: Theory, Media Fabrication, and Recording Performance , 2015, IEEE Transactions on Magnetics.

[37]  Eleni Drinea,et al.  Directly Lower Bounding the Information Capacity for Channels With I.I.D. Deletions and Duplications , 2010, IEEE Transactions on Information Theory.

[38]  Madhu Sudan,et al.  Synchronization Strings: List Decoding for Insertions and Deletions , 2018, ICALP.

[39]  Suhas N. Diggavi,et al.  Models and information-theoretic bounds for nanopore sequencing , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[40]  Olgica Milenkovic,et al.  Portable and Error-Free DNA-Based Data Storage , 2016, Scientific Reports.

[41]  Zhengzhong Jin,et al.  Deterministic Document Exchange Protocols, and Almost Optimal Binary Codes for Edit Errors , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[42]  Jehoshua Bruck,et al.  Duplication-correcting codes for data storage in the DNA of living organisms , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[43]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[44]  Rafail Ostrovsky,et al.  Low distortion embeddings for edit distance , 2005, STOC '05.

[45]  Vahid Tarokh,et al.  Bounds on the Capacity of Discrete Memoryless Channels Corrupted by Synchronization and Substitution Errors , 2012, IEEE Transactions on Information Theory.

[46]  G. Church,et al.  Next-Generation Digital Information Storage in DNA , 2012, Science.

[47]  Sampath Kannan,et al.  Reconstructing strings from random traces , 2004, SODA '04.

[48]  Kuei-Hung Shen,et al.  Racetrack Memory: A high-performance, low-cost, non-volatile memory based on magnetic domain walls , 2011, 2011 International Electron Devices Meeting.

[49]  Jeffrey D. Ullman,et al.  On the capabilities of codes to correct synchronization errors , 1967, IEEE Trans. Inf. Theory.

[50]  Tolga M. Duman,et al.  An upper bound on the capacity of non-binary deletion channels , 2013, 2013 IEEE International Symposium on Information Theory.

[51]  Wojciech Szpankowski,et al.  Fundamental Bounds for Sequence Reconstruction From Nanopore Sequencers , 2016, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[52]  Dario Fertonani,et al.  Novel Bounds on the Capacity of the Binary Deletion Channel , 2010, IEEE Transactions on Information Theory.

[53]  Tolga M. Duman,et al.  Achievable Rates for Noisy Channels With Synchronization Errors , 2014, IEEE Transactions on Communications.

[54]  Rafail Ostrovsky,et al.  Fuzzy Extractors: How to Generate Strong Keys from Biometrics and Other Noisy Data , 2004, SIAM J. Comput..

[55]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[56]  Farzad Farnoud,et al.  Small-sample distribution estimation over sticky channels , 2009, 2009 IEEE International Symposium on Information Theory.

[57]  Aleksandar Kavcic,et al.  Trellis based lower bounds on capacities of channels with synchronization errors , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[58]  Michael Mitzenmacher,et al.  Improved Lower Bounds for the Capacity of i.i.d. Deletion and Duplication Channels , 2007, IEEE Transactions on Information Theory.

[59]  Michael Mitzenmacher,et al.  A Survey of Results for Deletion Channels and Related Synchronization Channels , 2008, SWAT.

[60]  Amos Lapidoth,et al.  On the Capacity of the Discrete-Time Poisson Channel , 2009, IEEE Transactions on Information Theory.

[61]  Tongxin Li.  Maximum Likelihood Upper Bounds on the Capacities of Discrete Information Stable Channels , 2018, 2018 IEEE Information Theory Workshop (ITW).

[62]  Michael Mitzenmacher,et al.  On the zero-error capacity threshold for deletion channels , 2011, 2011 Information Theory and Applications Workshop.

[63]  Kenji Yasunaga,et al.  On the List Decodability of Insertions and Deletions , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[64]  Marco Dalai.  A new bound on the capacity of the binary deletion channel with high deletion probabilities , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[65]  Khaled Abdel-Ghaffar.  Capacity per unit cost of a discrete memoryless channel , 1993 .

[66]  Mahdi Cheraghchi,et al.  Sharp Analytical Capacity Upper Bounds for Sticky and Related Channels , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[67]  Venkatesan Guruswami,et al.  An Improved Bound on the Fraction of Correctable Deletions , 2015, IEEE Transactions on Information Theory.

[68]  Venkatesan Guruswami,et al.  Coding against deletions in oblivious and online models , 2018, SODA.

[69]  Alfonso Martinez,et al.  Spectral efficiency of optical direct detection , 2007 .

[70]  Masakazu Jimbo,et al.  An Iteration Method for Calculating the Relative Capacity , 1979, Inf. Control..

[71]  Kannan Ramchandran,et al.  Achievable Rates for Channels With Deletions and Insertions , 2011, IEEE Transactions on Information Theory.

[72]  Mladen Kovacevic.  Zero-Error Capacity of Duplication Channels , 2019, IEEE Transactions on Communications.

[73]  Vladimir I. Levenshtein,et al.  Efficient reconstruction of sequences , 2001, IEEE Trans. Inf. Theory.

[74]  Michael Mitzenmacher,et al.  On Lower Bounds for the Capacity of Deletion Channels , 2006, IEEE Transactions on Information Theory.

[75]  Masoud Ardakani,et al.  On the Capacity of Duplication Channels , 2013, IEEE Transactions on Communications.

[76]  Aleksandar Kavcic,et al.  Insertion/deletion channels: reduced-state lower bounds on channel capacities , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[77]  R. Gallager.  SEQUENTIAL DECODING FOR BINARY CHANNELS WITH NOISE AND SYNCHRONIZATION ERRORS , 1961 .

[78]  Paul H. Siegel,et al.  On the Capacity of Channels With Timing Synchronization Errors , 2013, IEEE Transactions on Information Theory.

[79]  Michael Langberg,et al.  A Characterization of the Number of Subsequences Obtained via the Deletion Channel , 2015, IEEE Transactions on Information Theory.

[80]  Bane V. Vasic,et al.  Coding for Correcting Insertions and Deletions in Bit-Patterned Media Recording , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[81]  Bernhard Haeupler,et al.  Synchronization strings: codes for insertions and deletions approaching the Singleton bound , 2017, STOC.

[82]  Jun Hu,et al.  Achievable information rates for channels with insertions, deletions, and intersymbol interference with i.i.d. inputs , 2010, IEEE Transactions on Communications.

[83]  Michael Mitzenmacher,et al.  Capacity Bounds for Sticky Channels , 2008, IEEE Transactions on Information Theory.

[84]  Venkatesan Guruswami,et al.  Deletion Codes in the High-Noise and High-Rate Regimes , 2014, IEEE Transactions on Information Theory.

[85]  J. Wolfowitz.  The coding of messages subject to chance errors , 1957 .

[86]  Suhas N. Diggavi,et al.  Capacity Upper Bounds for the Deletion Channel , 2007, 2007 IEEE International Symposium on Information Theory.

[87]  George S. Lueker,et al.  Improved bounds on the average length of longest common subsequences , 2003, JACM.

[88]  Joel G. Smith,et al.  The Information Capacity of Amplitude- and Variance-Constrained Scalar Gaussian Channels , 1971, Inf. Control..

[89]  Aleksandar Kavcic,et al.  Bounds on mutual information rates of noisy channels with timing errors , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[90]  Mahdi Cheraghchi.  Capacity upper bounds for deletion-type channels , 2018, STOC.

[91]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[92]  Jian Ma,et al.  DNA-Based Storage: Trends and Methods , 2015, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[93]  Alexandr Andoni,et al.  The Computational Hardness of Estimating Edit Distance , 2010 .

[94]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .