Searching the World's Herbaria: A System for Visual Identification of Plant Species

We describe a working computer vision system that aids in the identification of plant species. A user photographs an isolated leaf on a blank background, and the system extracts the leaf shape and matches it to the shape of leaves of known species. In a few seconds, the sys- tem displays the top matching species, along with textual descriptions and additional images. This system is currently in use by botanists at the Smithsonian Institution National Museum of Natural History. The primary contributions of this paper are: a description of a working com- puter vision system and its user interface for an important new applica- tion area; the introduction of three new datasets containing thousands of single leaf images, each labeled by species and verified by botanists at the US National Herbarium; recognition results for two of the three leaf datasets; and descriptions throughout of practical lessons learned in constructing this system.

[1]  Enrique Vidal-Ruiz,et al.  An algorithm for finding nearest neighbours in (approximately) constant average time , 1986, Pattern Recognit. Lett..

[2]  R. J. Pankhurst,et al.  Practical Taxonomic Computing , 1991 .

[3]  Enrique Vidal,et al.  New formulation and improvements of the nearest-neighbour approximating and eliminating search algorithm (AESA) , 1994, Pattern Recognit. Lett..

[4]  M. Edwards,et al.  The potential for computer-aided identification in biodiversity research. , 1995, Trends in ecology & evolution.

[5]  Josef Kittler,et al.  Reliable Classification of Chrysanthemum Leaves through Curvature Scale Space , 1997, Scale-Space.

[6]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[8]  Takeshi Saitoh,et al.  Automatic recognition of wild flowers , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[9]  Ivan Poupyrev,et al.  Virtual object manipulation on a table-top AR environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[10]  P. Bryan Heidorn,et al.  A Tool for Multipurpose Use of Online Flora and Fauna: The Biological Information Browsing Environment (BIBE) , 2001, First Monday.

[11]  Oskar Söderkvist,et al.  Computer Vision Classification of Leaves from Swedish Trees , 2001 .

[12]  Benjamin B. Bederson,et al.  PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps , 2001, UIST '01.

[13]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[14]  Ronen Basri,et al.  Texture segmentation by multiscale aggregation of filter responses and shape elements , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  R. Stevenson,et al.  Electronic Field Guides and User Communities in the Eco-informatics Revolution , 2003 .

[16]  Zhiyong Wang,et al.  Shape based leaf image retrieval , 2003 .

[17]  Benjamin B. Bederson,et al.  Toolkit design for interactive structured graphics , 2004, IEEE Transactions on Software Engineering.

[18]  Sadegh Abbasi,et al.  Matching shapes with self-intersections:application to leaf classification , 2004, IEEE Transactions on Image Processing.

[19]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Sean White,et al.  Virtual Vouchers: Prototyping a Mobile Augmented Reality User Interface for Botanical Species Identification , 2006, 3D User Interfaces (3DUI'06).

[21]  Sean White,et al.  First steps toward an electronic field guide for plants , 2006 .

[22]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Sean White,et al.  Designing a mobile user interface for automated species identification , 2007, CHI.

[24]  Haibin Ling,et al.  Shape Classification Using the Inner-Distance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Joshua D. Schwartz,et al.  Hierarchical Matching of Deformable Shapes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.