Accurate automatic medical image segmentation technology plays an important role for the diagnosis and treatment of brain tumor. However, existing methods based on outstanding 2.5D and 3D segmentation strategies are time-consumption and hardware-consumption while ensuring high accuracy. In order to reduce the high demand for automatic segmentation of tumor images and avoid the noise interference in a single input image, we propose an end-to-end zero-shot CNN segmentation method. Our method only utilizes two adjacent images, instead of the target image, as the input data of deep neural network to predict the brain tumor area in the target image. Avoiding noise interference in the target image, this method makes full use of the spatial context feature between adjacent slices in order to obtain accurate zero-shot segmentation results. We compare with the state-of-the-art segmentation frameworks on the same benchmark and notice that our method has strong competitiveness.