Hard Sample Matters a Lot in Zero-Shot Quantization