Density matrix formalism for coupled dynamical systems