Topological insulators and fractional quantum Hall effect on the ruby lattice