General theory of effective Hamiltonians