Quantum field theory of classically unstable Hamiltonian dynamics