This paper describes recent work on acoustic segmentation and phonetic classification as part of an effort in speech understanding system development. The signal representation is based on an auditory model that incorporates known properties of the human auditory system, including critical‐band filtering, saturation, adaptation, forward masking, and synchrony detection. Acoustic landmarks are determined using a measure of local similarity. These landmarks are embedded in a multilevel structure in which information ranging from coarse to fine is represented in an organized fashion. An analysis of the acoustic structure, using 500 utterances from 100 different talkers, shows that it captures over 96% of the acoustic‐phonetic events of interest with an insertion rate of less than 5%. Phonetic classification is achieved by defining a set of generic property detectors based on the knowledge of acoustic phonetics. The settings of the parameters are obtained by a search procedure, using a large body of training ...