Topological phase transition and nontrivial thermal Hall signatures in honeycomb lattice magnets