The retrieval of land surface temperature (LST) using thermal infrared (TIR) data is important in many applications. However, TIR data usually suffer from low spatial resolution. We introduce a novel subpixel LST estimation model using the information-transfer-based adaptive ensemble extreme learning machine (IT-AEELM). The proposed method constructs a reliable relationship between subpixel LST and the input high-resolution visible and near-infrared (VNIR) data, short-wave infrared (SWIR) data, and low-resolution TIR data. Based on a detailed analysis of different ground objects, we divide the input data into multiple subsets. Instead of using consistent land surface parameters (LSPs), we utilize different LSPs to characterize the land surface properties in each subset. The VNIR-SWIR-LSPs data and the low-resolution LST are used to train a novel IT-AEELM network, where a feedback ensemble learning scheme is introduced to effectively remove inaccurate estimates. The main difference of the model against existing methods is that it builds a robust architecture at different spatial scales, which provides benefits including lower demand for training data, more rapid and accurate acquisition of subpixel LST, and better adaption to heterogeneous land surface. Numerical experiments demonstrate that the proposed method significantly improves the accuracy of subpixel LST compared with the state-of-the-art algorithms.