Flatness of Tensor Products and Semi-Rigidity for $C_2$-cofinite Vertex Operator Algebras II