Boosting quantum key distribution via the end-to-end physical control