Isolated photon-hadron production in high energy $pp$ and $pA$ collisions at RHIC and LHC