Quantifying the colour-dependent stochasticity of large-scale structure