Exact Solutions of a Deep Linear Network