Opportunities for Natural Language Processing in Qualitative Engineering Education Research: Two Examples

This research full paper proposes opportunities to expand qualitative and textual data analysis using natural language processing (NLP), and demonstrates opportunities to use NLP in engineering education work in our presentation of two NLP-based projects as examples of how NLP can be used. The discipline of engineering education frequently employs qualitative data analysis techniques, but fundamentally, the analysis of large corpora of textual documents is limited by researcher’s time. In this paper, we present a brief review literature of how NLP has been used in other qualitative research fields. This portion of the paper is aimed to provide a clear description of NLP for those unfamiliar with machine learning and natural language processing methods. The second part of the paper will provide two brief examples of how NLP is being employed in our research group. Example 1 is a study of engineering résumés, with the intention of being able to calculate the “disciplinary discourse density” based off the engineering language presented in engineering résumés, a technique validated in prior qualitative studies by hand. Example 2 is a genre analysis of engineering literature reviews, seeking to understand the ways in which sentences linguistically build into arguments, such that the task of writing literature reviews might be demystified. This paper will have a methodological impact for researchers attempting to use NLP methods to analyze qualitative data, articulating the opportunities and barriers in using these methods for engineering education research.

[1]  Shuang Song,et al.  Triangulation of Indicators of Successful Student Design Teams , 2006 .

[2]  Mary McCall,et al.  Quantification of Engineering Disciplinary Discourse in Résumés: A Novel Genre Analysis With Teaching Implications , 2018, IEEE Transactions on Professional Communication.

[3]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[4]  Jacob Grohs,et al.  Leveraging institutional data to understand student perceptions of teaching in large engineering classes , 2017, 2017 IEEE Frontiers in Education Conference (FIE).

[5]  Mary McCall,et al.  A degree is not enough: Promoting engineering identity development and professional planning through the teaching of engineering résumé writing , 2016, 2016 IEEE Frontiers in Education Conference (FIE).

[6]  L. Kof.  NATURAL LANGUAGE PROCESSING FOR REQUIREMENTS ENGINEERING : APPLICABILITY TO LARGE REQUIREMENTS DOCUMENTS , 2004 .

[7]  Catherine G. P. Berdanier,et al.  Optimizing a genre analysis framework to investigate engineering literature reviews , 2017, 2017 IEEE International Professional Communication Conference (ProComm).

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[10]  Ahmet Fevzi Baba,et al.  Intelligent Fuzzy Assessment System for English Academic Writing in Engineering Education , 2015 .

[11]  Jonathan Cagan,et al.  THE MEANING OF "NEAR" AND "FAR": THE IMPACT OF STRUCTURING DESIGN DATABASES AND THE EFFECT OF DISTANCE OF ANALOGY ON DESIGN OUTPUT , 2012 .

[12]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[13]  Manfred K. Warmuth,et al.  THE CMU SPHINX-4 SPEECH RECOGNITION SYSTEM , 2001 .

[14]  Jonathan Cagan,et al.  The Role of Functionality in the Mental Representations of Engineering Students: Some Differences in the Early Stages of Expertise , 2006, Cogn. Sci..