Matrix factorizations of determinants and permanents