Optimal design of transmitter and receiver filters with mixed performance objectives

Nonsmooth optimization is applied to the design of transmitter and receiver filters for data transmission. This optimization method allows tradeoffs between several performance measures-ISI, transmitter filter average power output, and receiver filter power gain. An FCC mask constraint on the transmitter filter is imposed. An example illustrates the design method.