We propose a scheme of loss-resilient entanglement swapping between two distant parties via an imperfect optical channel. In this scheme, two copies of hybrid entangled states are prepared and the continuous-variable parts propagate through lossy media. In order to perform successful entanglement swapping, several different measurement schemes are considered for the continuous-variable parts such as single-photon detection for ideal cases and a homodyne detection for practical cases. We find that the entanglement swapping using hybrid states with small amplitudes offers larger entanglement than the discrete-variable entanglement swapping in the presence of large losses. Remarkably, this hybrid scheme still offers excellent robustness of entanglement to the detection inefficiency. Thus, the proposed scheme could be used for the practical quantum key distribution in hybrid optical states under photon losses.