Cohomological rigidity for toric Fano manifolds of small dimensions or large Picard numbers