Constraining weak annihilation using semileptonic D decays