Lower estimates near the origin for functional calculus on operator semigroups