Data Mining Solution for Assessing Brazilian Secondary School Quality Based on ENEM and Census Data

This paper presents a data mining solution for assessing the quality of Brazilian private secondary schools based on the official school survey and students tests. Following the CRISP-DM method, after the problem interpretation and modeling, these two data sources yearly collected have been transformed to the school granularity level embedding data and expert´s knowledge and have been integrated in a single data set with the national school code as primary key. Further transformations on the joint data set embedded additional knowledge and made the format compatible with the artificial intelligence techniques applied for knowledge extraction. Logistic regression was applied for producing a propensity score for good schools, decision tree applied for extracting the sequential decision making a human would follow and rules were induced for supporting the explanation of a decision based on the score. The AUC_ROC and Max_KS were used for assessing the propensity score performance and, coverage, confidence and lift were used for assessing the quality of the rules induced by the A Priori algorithm, together with the human knowledge available on the literature. The results showed that this domain-driven data mining approach was successful in modeling the problem and validating educational public policies.