Matrices with prescribed rows and invariant factors