Least Squares Problems

Here we solve linear systems Ax = b that do not have a solution. If b is not in the column space of A, there is no x such that Ax = b. The best we can do is to find a vector y that brings left and right hand side of the linear system as close as possible, in other words y is chosen to make the distance between Ay and b as small as possible. That is, we want to minimize the distance ‖Ax − b‖2 over all x, and distance will again be measured in the twonorm.