A lower bound on the bit-error-rate resulting from mismatched viterbi decoding

A lower bound is derived on the bit error rate that results when a rate l/n convolutionally encoded binary data stream is transmitted over a noisy symmetric channel, and is then decoded using a mismatched Viterbi decoder, i.e., a Viterbi decoder that performs maximum-likelihood sequence estimation using possibly incorrect branch metrics. The branch metrics are assumed to be symmetric, but are generally differrent from the log-likelihood function. The proposed bound is expressed in terms of the pairwise error probability for a given error event normalized by the sum of the events length and the code's memory. While the bound can be somewhat loose for medium signal-to-noise ratios, it is usually asymptotically tight at high signal-to-noise ratios. For the special case where the branch metrics employed are equal to the log-likelihood function our lower bound does not coincide with the standard bound that was derived by Forney and Mazo. It falls short by a multiplicative constant that depends on the code's memory and the shortest error event of minimum Hamming weight. We apply our lower bound to the study of convolutionally encoded direct-sequence spread-spectrum communication with Laplacian noise and show that nearest-neighbor decoding, which is optimal for Gaussian noise, is sub-optimal and asymptotically (for high processing gain) results in a 3 dB loss in performance when compared with the optimal maximum-likelihood decoder. Extensions to rate c/n convolutional encoders, general trellis encoders and to symmetric channels with memory are also presented.