A Continuum Limit of Matrix Inverse Problems