The operator theory of the pseudo-inverse I. Bounded operators