No guaranteed neutrino astronomy without (enough) taus