A Novel Vision-based Approach for Dietary Assessment using Deep Learning View Synthesis

Dietary assessment system has proven as an effective tool to evaluate the eating behavior of patients suffering from diabetes and obesity. To assess the dietary intake, the traditional method is to carry out a 24-hour dietary recall (24HR), a structured interview aimed at capturing information on food items and portion size consumed by participants. However, unconscious biases are developed easily due to individual's subjective perception in this self-reporting technique which may lead to inaccuracy. Thus, this paper proposed a novel vision-based approach for estimating the volume of food items based on deep learning view synthesis and depth sensing techniques. In this paper, a point completion network is applied to perform 3D reconstruction of food items using a single depth image captured from any convenient viewing angle. Compared to previous approaches, the proposed method has addressed several key challenges in vision-based dietary assessment, such as view occlusion and scale ambiguity. Experiments have been carried out to examine this approach and showed the feasibility of the algorithm in accurate estimation of food volume.