Queen Jane Approximately: Enabling Efficient Neural Network Inference with Context-Adaptivity