Understanding Plane Geometry Problems by Integrating Relations Extracted from Text and Diagram

Understanding problems is a critical and hard step for solving plane geometry problems. This paper presents a method for understanding plane geometry problems by integrating the information from text and diagram two modalities. Then high-confidence geometric relations are extracted for problem understanding through integrating the information separately extracted from text and diagram. And this paper also presents a function that can visually encode the extracted relations into the diagram to interactively present the visual effects of problem understanding results. The geometric relations are mined from the diagram after visual primitives are detected. The syntax-semantics (S\( ^{2} \)) model method is adopted to extract geometric relations from the text. And an integration process is used to couple the information from text and diagram to obtain the high-confidence geometric relations. The experimental results show that the proposed method can mine geometric relations in high accuracy and it can understand some problems that cannot be understood by using text only or by using diagram only.