A scientific writing pedagogy and mixed methods assessment for engineering education using open-coding and multi-dimensional scaling

Assessments of new pedagogical practices usually rely on instructor oriented surveys and questionnaires to measure student perceptions of teaching methods; however, fixed response categories in structured questionnaires might bias results. This paper demonstrates a mixed methods approach using open and multi-dimensional scaling (MDS) for a student-oriented exploratory analysis and visualization of perceptions of teaching methods. A scientific writing and research methodology course is a required course for first year PhD students in software engineering and geo-infomatics at Wuhan University, China. PhD students attending this course came from countries whose first language is not English, and from a variety of software engineering and geo-infomatics domains. The problem therefore, was to elicit un-mediated perceptions of course assignment, reduce and generalize the resulting data for interpretation. A graphical visualization of themes emerging in student responses to two open-ended questions about an assignment provided a basis for inferring student interests and needs. In this course, an assessment of a task oriented problem-solving experience was implemented through a mixed method strategy incorporating qualitative methods, exploratory data mining techniques, and cartographic visualization. The visualization shows that participating students generally perceived the exercise as challenging, helped them understand journal requirements, and develop ways to survey texts to extract information. The results also suggested that this consensus breaks down in terms of each participant’s own goals, domain, and research interests. Unstructured questions, open coding, and MDS visualization might also prove to be helpful in the process of devising and assessing other student centered pedagogies.