Theory of the electron spin relaxation in graphene on the SiO$_2$ substrate is developed. Charged impurities and polar optical surface phonons in the substrate induce an effective random Bychkov-Rashba-like spin-orbit coupling field which leads to spin relaxation by the D'yakonov-Perel' mechanism. Analytical estimates and Monte Carlo simulations show that the corresponding spin relaxation times are between micro- to milliseconds, being only weakly temperature dependent. It is also argued that the presence of adatoms on graphene can lead to spin lifetimes shorter than nanoseconds.