Vulnerabilities and imperfections of single-photon detectors have been shown to compromise security for quantum key distribution (QKD). The measurement-device-independent QKD (MDI-QKD) appears to be the most appealing solution to solve the issues. However, in practice one faces severe obstacles of having significantly lower key generation rate, difficult two photon interferences, and remote synchronization etc. In this letter, we propose a highly efficient and simple quantum key distribution scheme to remove all of these drawbacks. Our proposal can be implemented with only small modifications over the standard decoy BB84 system. Remarkably it enjoys both the advantages of high key generation rate (being almost two orders of magnitude higher than that based on conventional MDI-QKD) comparable to the normal decoy system, and security against any detector side channel attacks. Most favorably one can achieve complete Bell state measurements with resort to single photon interference, which reduces significantly experimental costs. Our approach enables utilization of high speed and efficient secure communication, particularly in real-life scenario of both metropolitan and intercity QKD network, with an attack free fashion from arbitrary detector side channels.