This paper aims to study the construction method of a three-dimensional temperature distribution reconstruction system based on depth and thermal infrared information. Initially, the traditional calibration methods were not straightforward because depth and thermal infrared cameras were not sensitive to color calibration plates. Therefore, a depth and thermal infrared camera calibration board is designed to perform depth and thermal infrared camera calibration. Secondly, the three-dimensional temperature distribution is built based on spatial affine transformation matching. The Local feature descriptor is proposed. In addition, the three-dimensional temperature distribution is calculated by local feature matching. Finally, the optimization of 3D temperature distribution reconstruction is completed based on autoregressive (AR) model. Experimental results show that a three-dimensional temperature distribution model can be built accurately based on this method with strong robustness.