Visible and infrared imaging spectroscopy have greatly revolutionized our understanding of the diversity of minerals on Mars. Characterizing the mineral distribution on Mars is essential for understanding its geologic evolution and past habitability. The traditional handcrafted spectral index could be ambiguous as it may denote broad mineralogical classes, making this method unsuitable for definitive mineral investigation. In this work, the target detection technique is introduced for specific mineral mapping. We have developed a new subpixel mineral detection method by joining the Hapke model and spatially adaptive sparse representation method. Additionally, an iterative background dictionary purification strategy is proposed to obtain robust detection results. Laboratory hyperspectral image containing Mars Global Simulant and serpentine mixtures was used to evaluate and tailor the proposed method. Compared with the conventional target detection algorithms, including constrained energy minimization, matched filter, hierarchical constrained energy minimization, sparse representation for target detection, and spatially adaptive sparse representation method, the proposed algorithm has a significant improvement in accuracy about 30.14%, 29.67%, 29.41%, 9.13%, and 8.17%, respectively. Our algorithm can detect subpixel serpentine with an abundance as low as 2.5% in laboratory data. Then the proposed algorithm was applied to two well-studied Compact Reconnaissance Imaging Spectrometer for Mars images, which contain serpentine outcrops. Our results are not only consistent with the spatial distribution of Fe/Mg phyllosilicates derived by spectral indexes, but also denote what the specific mineral is. Experimental results show that the proposed algorithm enables the search for subpixel, low-abundance, and scientifically valuable mineral deposits.