Matrix Differential Calculus Jan R. Magnus and Heinz Neudecker John Wiley and Sons, 1988Linear Structures Jan R. Magnus Charles Griffin and Co., 1988

One of the more onerous tasks of multivariate statistical analysis is that of finding the first-order and second-order partial derivatives of complicated likelihood functions involving vectors or matrices as their arguments. Two decades ago, the usual way of tackling such problems was to resort to scalar differential calculus. Thus, a function expressed compactly in terms of matrices would be expanded in scalar form, differentiated by scalar methods, and then reassembled in the desired matrix form (for examples, see Anderson [1, p. 47] and Rao [19, p. 448]). This resort to scalar methods was the consequence of an hiatus in matrix theory that threatened to impede the development of multivariate statistical analysis. Efforts to develop a theory of matrix differential calculus were heralded by the seminal article of Dwyer and MacPhail [5] and were given further impetus some twenty years later by Dwyer [4]. These authors were faced with the problem differentiating an arbitrary function Y = Y(X), where Y = [yki] and X = [Xjj] are matrices of any order. However, they avoided the difficulties inherent in a fully fledged theory of matrix differential calculus by confining their attention to the component derivatives dykl/dX and dY/dXjj. Although the derivative dY/dxtj proved easier to handle, it appeared that the derivative dykI/dX was the more useful in multivariate analysis. By developing a general theory that linked the two forms of derivative, Dwyer and MacPhail were able to obtain their derivatives in one form and convert them into the other form for practical use. Since the appearance of the article by Dwyer, there has been a rapid growth in the literature on matrix differential calculus and on the related topics of the algebra of Kronecker products and the vectorization operator.