Images compressed by current lossy compression techniques suffer from distortion that is uniformly distributed spatially and spectrally. We demonstrate that classification accuracy changes as a function of pixel class and spectral location (spectral bands) of the distortion and that the uniform distribution of distortion is therefore not an optimal approach to controlling distortion within an image. We show that a superior approach involves locating areas within the image whose classification accuracies are relatively insensitive to distortion and limiting the application of distortion to these areas. By following this approach, we show that substantial levels of compression-induced distortion can be tolerated without a significant reduction in subsequent classification accuracy. Hyperspectral images are prime candidates for data compression due to their inherent size. Lossy compression algorithms are attractive as they typically provide the most impressive levels of compression (compression ratio) but result in some distortion of the original image. The acceptability of this distortion depends on the end use of the data set that, in the case of hyperspectral images, invariably involves the use of computer-based tools to process the images into a set of classes representing the ground coverage or conditions present in the data. Compression-induced distortion tends to reduce the accuracy associated with the classification process, but the relationship between distortion and classification accuracy varies across different classes of data within the same image and is not particularly predictable. We demonstrate an alternative to the uniform application of distortion during compression that aims to locate spectral and spatial areas within an image where sensitivity to distortion is likely to be reduced. We then restrict the application of the compression-induced distortion to these areas of low sensitivity and show that the subsequent classification accuracies are superior to uniformly distorted images.