Low-bit quantization is an effective way to save storage and computing resources. However, the accuracy of the low-bit quantization model is low. The loss of the quantization model’s precision is the major cause. Creating a quantization loss function is the most direct and effective method to optimize the quantization error. However, Because of the underivable quantization function, the optimized quantization loss function is difficult to be designed. To solve these problems, we propose a learnable quantization loss function base on expectation for minimizing quantization error. Extensive experiments on image classification show that our method outperforms the state-of-the-art techniques for all bit widths. The accuracy of our method is much higher than that of the existing methods. Significantly, the accuracy of ResNet18 with 1-bit weight and 1-bit activation is 5.18% higher than that of the current state-of-the-art binary network on the ImageNet classification task.