Supplementary Materials: Controllable Orthogonalization in Training DNNs