Codebook optimization in vector quantization