Weak lensing convergence peaks are a promising tool to probe nonlinear structure evolution at late times, providing additional cosmological information beyond second-order statistics. Previous theoretical and observational studies have shown that the cosmological constraints on ${\mathrm{\ensuremath{\Omega}}}_{m}$ and ${\ensuremath{\sigma}}_{8}$ are improved by a factor of up to $\ensuremath{\approx}2$ when peak counts and second-order statistics are combined, compared to using the latter alone. We study the origin of lensing peaks using observational data from the $154\text{ }\text{ }{\mathrm{deg}}^{2}$ Canada-France-Hawaii Telescope Lensing Survey. We found that while high peaks (with height $\ensuremath{\kappa}g3.5{\ensuremath{\sigma}}_{\ensuremath{\kappa}}$, where ${\ensuremath{\sigma}}_{\ensuremath{\kappa}}$ is the rms of the convergence $\ensuremath{\kappa}$) are typically due to one single massive halo of $\ensuremath{\approx}1{0}^{15}{M}_{\ensuremath{\bigodot}}$, low peaks ($\ensuremath{\kappa}\ensuremath{\lesssim}{\ensuremath{\sigma}}_{\ensuremath{\kappa}}$) are associated with constellations of 2\char21{}8 smaller halos ($\ensuremath{\lesssim}1{0}^{13}{M}_{\ensuremath{\bigodot}}$). In addition, halos responsible for forming low peaks are found to be significantly offset from the line of sight towards the peak center (impact parameter $\ensuremath{\gtrsim}$ their virial radii), compared with $\ensuremath{\approx}0.25$ virial radii for halos linked with high peaks, hinting that low peaks are more immune to baryonic processes whose impact is confined to the inner regions of the dark matter halos. Our findings are in good agreement with results from the simulation work by Yang et al. [Phys. Rev. D 84, 043529 (2011)].