In this article, we aim to improve the performance of synthetic aperture radar (SAR) ship detection under complex conditions. The complex backgrounds are commonly encountered for high-resolution (HR) SAR ship detection data set, and they greatly influence the detection performance of ships. In recent years, deep neural networks (DNNs) have made substantial improvements on detection by adopting data augmentation. However, the improvement is limited since the models are sensitive to noise. To address this problem, a Fast Saliency-based Filtering algorithm (FSF) is proposed to filter out interference information. The FSF method is inspired by the filtering mechanisms of the human brain, which help people filter out target-irrelevant information fast to better extract target-relevant information. The FSF includes two parts of the bottom-up process and the top-down process. The bottom-up process is used to extract a saliency map of an input image, and the other one is used to filter out target-irrelevant information based on the saliency map. The FSF can be a front-end preprocessing module of DNNs to fast filter out target-irrelevant information and obtain a primary priority map of an input image. Experimental results demonstrate that our brain-inspired FSF method obtains obvious improvement of detection performance on AIR-SARShip-1.0.