The Origin of the Colour Bimodality in the Scatter of the Stellar-to-Halo Mass Relation