Foundations and applications of quantum kinetic theory