Accurate and reliable indoor location estimate is crucial for many Internet of Things (IoT) applications in the era of smart buildings. However, the positioning accuracy and security of the existing positioning works cannot meet the demands in the large-scale smart buildings scenarios covering multiple multi-floor buildings. Therefore, in this paper, we focus on the reliable and accurate localization under multi-building and multi-floor environments. We propose two novel designs, including a two-step reliable feature selector, and a multi-task collaborative positioning model. Firstly, we design a two-step reliable feature selector based on a access point (AP) confidence model and manifold learning, to help select the most representative and reliable fingerprint features. Secondly, we propose a multi-task cooperative positioning model, which consists of a multi-scale feature fusion module to adaptively fuse multi-scale features and a multi-task joint learning module to effectively constrain the cumulative error of multi-scale position. Finally, based on above two, we propose a reliable multi-building and multi-floor localization method (RMBMFL), which can achieve accurate and reliable location estimates with low computational complexity in a smart building complex. We did real-world experiments in a 20,000 m2 site that covers three multi-story buildings to evaluate the performance of the proposed RMBMFL. Experimental results show that RMBMFL achieves a building identification accuracy and a floor identification accuracy of 99%, and a room-level indoor localization with an average positioning error within 2m, and outperforms state-of-the-art solutions.