Integrating leaf and flower by local discriminant CCA for plant species recognition

Abstract Plant species recognition using a single organ, such as flower and leaf, is not sufficiently reliable, because different species may have very similar flowers or leaves, while the same species may have rather different flowers or leaves. Combining leaves and flowers to recognize plant species can produce positive results. Based on multi-modal learning scheme, an automatic plant species recognition method is proposed by combining leaves and flowers of plant. In the method, a modified local discriminant canonical correlation analysis (MLDCCA) is designed by incorporating the idea of local discriminant embedding (LDE) into canonical correlation analysis (CCA). Firstly, two neighbor graphs are constructed based on the exploration of the manifold that the input data lie on. Then, two projection matrices for dimensionality reduction are obtained by making the within-class neighbor samples most correlated and between-class neighbor samples least correlated, and meanwhile keeping the correlation between leaves and flowers of the same species maximum. Finally, 1-nearest neighbor classifier with geodesic distance is used to recognize the plant species. MLDCCA is a powerful supervised multi-modal dimensional reduction method which can extract the discriminant features from two plant organs, meanwhile preserve the discriminant information and the data structure well. Experimental results on a real leaf and flower image dataset validate the effectiveness of the proposed method.