High-throughput 3D modelling of plants for phenotypic analysis

In this paper we propose a twin mirror-based system for reconstructing 3D models of real plants for subsequent phenotypic analysis. The method is based on the visual hull concept: multiple reflections of the object from the mirrors give different views of the object and are interpreted as taken from virtual cameras. The epipolar geometry of the object and its four reflections is determined without relying on information of the positions of the camera and mirrors. This alleviates the usual camera calibration step. Two simultaneous images of object mirror scene give ten different and simultaneous views of the plant, without requiring any plant or camera movement. Visual hull algorithms are sensitive to segmentation of the object from the scene. We propose a novel machine learning approach to segment a plant from its background. The plant colours are represented using a Gaussian mixture model (GMM), while the background colours are represented by a separate GMM, learnt using an Expectation Maximisation (EM) algorithm. A Bayes classification rule that satisfies the Neymann-Pearson criteria is used to classify the pixels and thus segment the five plant silhouette from each image. We show results of 3D models of wheat, grass, and a lavender shoot reconstructed using the proposed segmentation and 3D visual hull method.