The deepest X-ray view of high-redshift galaxies: constraints on low-rate black-hole accretion