Multi-mode multi-stage vector quantization algorithm based on codeword characteristics