The toroidal dipole moments of natural molecules are hard to be detected so the artificial toroidal materials made by metamaterial attract more attentions. Metamaterial, the sub-wavelength artificial structures, can modulate reflection or transmission of light. The toroidal metamaterial can not only amplify the toroidal moment but also repress the electric and magnetic dipole so it can be used to study the properties of toroidal dipole moment. However, there are many limitations for the experiments, such as the lateral light is necessary to excite the toroidal response. Most of the toroidal dipole moments oscillate perpendicularly to the substrate, therefore it is difficult to couple it with other dipole moments and could be only excited in the microwave region. In this paper, we design a toroidal metamaterial consisting of dumbbell-shaped aperture and vertical split ring resonator (VSRR) vertically. The toroidal dipole moment of our metamaterial is excited in the optical region. The arrangement of our nanostructures is vertical instead of planar annular arrangement to reduce the size of the unit cell and increase the density of the toroidal dipole moment. Moreover, the direction of toroidal dipole moment is parallel to the substrate which can be used for the study of the coupling effect with other kinds of dipolar moments.