A large number of studies have demonstrated that depression patients have cognitive dysfunction. With recently developed brain functional imaging, studies have focused on changes in brain function to investigate cognitive changes. However, there is still controversy regarding abnormalities in brain functions or correlation between cognitive impairment and brain function changes. Thus, it is important to design an emotion-related task for research into brain function changes. We selected positive, neutral, and negative pictures from the International Affective Picture System. Patients with major depressive disorder were asked to judge emotion pictures. In addition, functional MRI was performed to synchronously record behavior data and imaging data. Results showed that the total correct rate for recognizing pictures was lower in patients compared with normal controls. Moreover, the consistency for recognizing pictures for depressed patients was worse than normal controls, and they frequently recognized positive pictures as negative pictures. The consistency for recognizing pictures was negatively correlated with the Hamilton Depression Rating Scale. Functional MRI suggested that the activation of some areas in the frontal lobe, temporal lobe, parietal lobe, limbic lobe, and cerebellum was enhanced, but that the activation of some areas in the frontal lobe, parietal lobe and occipital lobe was weakened while the patients were watching positive and neutral pictures compared with normal controls. The activation of some areas in the frontal lobe, temporal lobe, parietal lobe, and limbic lobe was enhanced, but the activation of some areas in the occipital lobe were weakened while the patients were watching the negative pictures compared with normal controls. These findings indicate that patients with major depressive disorder have negative cognitive disorder and extensive brain dysfunction. Thus, reduced activation of the occipital lobe may be an initiating factor for cognitive disorder in depressed patients.