As microRNAs (miRNAs) have been reported to be a type of novel high-value small molecule (SM) drug targets for disease treatments, many researchers are engaged in the field of exploring new SM-miRNA associations. Nevertheless, because of the high cost, adopting traditional biological experiments constrains the efficiency of discovering new associations between SMs and miRNAs. Therefore, as an important auxiliary tool, reliable computational models will be of great help to reveal SM-miRNA associations. In this article, we developed a computational model of sparse learning and heterogeneous graph inference for small molecule-miRNA association prediction (SLHGISMMA). Initially, the sparse learning method (SLM) was implemented to decompose the SM-miRNA adjacency matrix. Then, we integrated the reacquired association information together with the similarity information of SMs and miRNAs into a heterogeneous graph to infer potential SM-miRNA associations. Here, the main innovation of SLHGISMMA lies in the introduction of SLM to eliminate noises of the original adjacency matrix to some extent, which plays an important role in performance improvement. In addition, to assess SLHGISMMA' performance, four different kinds of cross-validations were performed based on two datasets. As a result, based on dataset 1 (dataset 2), SLHGISMMA achieved area under the curves of 0.9273 (0.7774), 0.9365 (0.7973), 0.7703 (0.6556), and 0.9241 ± 0.0052 (0.7724 ± 0.0032) in global leave-one-out cross-validation (LOOCV), miRNA-fixed local LOOCV, SM-fixed local LOOCV, and 5-fold cross-validation, respectively. Moreover, in the case study on three important SMs via removing their known associations, the results showed that most of the top 50 predicted miRNAs were confirmed by the database SM2miR v1.0 or the experimental literature.