Student Teamwork on Programming Projects: What can GitHub logs show us?

Teamwork, often mediated by version control systems such as Git and Apache Subversion (SVN), is central to professional programming. As a consequence, many colleges are incorporating both collaboration and online development environments into their curricula even in introductory courses. In this research, we collected GitHub logs from two programming projects in two offerings of a CS2 Java programming course for computer science majors. Students worked in pairs for both projects (one optional, the other mandatory) in each year. We used the students' GitHub history to classify the student teams into three groups, collaborative, cooperative, or solo-submit, based on the division of labor. We then calculated different metrics for students' teamwork including the total number and the average number of commits in different parts of the projects and used these metrics to predict the students' teamwork style. Our findings show that we can identify the students' teamwork style automatically from their submission logs. This work helps us to better understand novices' habits while using version control systems. These habits can identify the harmful working styles among them and might lead to the development of automatic scaffolds for teamwork and peer support in the future.

[1]  Hanna Järvenoja,et al.  Socio-emotional conflict in collaborative learning—A process-oriented case study in a higher education context , 2014 .

[2]  Jaakko Kurhila,et al.  Using Students' Programming Behavior to Predict Success in an Introductory Mathematics Course , 2013, EDM.

[3]  G. Salomon,et al.  When teams do not function the way they ought to , 1989 .

[4]  Greg Wilson,et al.  Mining student CVS repositories for performance indicators , 2005, MSR.

[5]  Tri Nguyen,et al.  Predictive Tool for Software Team Performance , 2016, 2016 23rd Asia-Pacific Software Engineering Conference (APSEC).

[6]  Peter Brusilovsky,et al.  Exploring Problem Solving Paths in a Java Programming Course , 2014, PPIG.

[7]  Christopher D. Hundhausen,et al.  The Normalized Programming State Model: Predicting Student Performance in Computing Courses Based on Programming Behavior , 2015, ICER.

[8]  James D. Herbsleb,et al.  Supporting Virtual Team Formation through Community-Wide Deliberation , 2017, Proc. ACM Hum. Comput. Interact..

[9]  M. Hoegl,et al.  Teamwork Quality and the Success of Innovative Projects , 2001 .

[10]  Frederick W. B. Li,et al.  No tests required: comparing traditional and dynamic predictors of programming success , 2014, SIGCSE.

[11]  Susan Brown Feichtner,et al.  Why Some Groups Fail: a Survey of Students' Experiences with Learning Groups , 1984 .

[12]  Paulo Blikstein,et al.  Using learning analytics to assess students' behavior in open-ended programming tasks , 2011, LAK.

[13]  Hao Xu,et al.  Assisting Instructor Assessment of Undergraduate Collaborative Wiki and SVN Activities , 2012, EDM.

[14]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[15]  Juan Enrique Ramos,et al.  Using TF-IDF to Determine Word Relevance in Document Queries , 2003 .

[16]  Christopher D. Hundhausen,et al.  Blending Measures of Programming and Social Behavior into Predictive Models of Student Achievement in Early Computing Courses , 2017, ACM Trans. Comput. Educ..

[17]  Hajimu Iida,et al.  A multiple-view analysis model of debugging processes , 2002, Proceedings International Symposium on Empirical Software Engineering.

[18]  P.K. Imbrie,et al.  Work in progress - a model to evaluate team effectiveness , 2005, Proceedings Frontiers in Education 35th Annual Conference.

[19]  S.G. Northrup,et al.  Multidisciplinary Teamwork Assessment: Individual Contributions and Interdisciplinary Interaction , 2006, Proceedings. Frontiers in Education. 36th Annual Conference.

[20]  Hyunjung Byun,et al.  Are high achievers successful in collaborative learning? An explorative study of college students’ learning approaches in team project-based learning , 2017 .

[21]  Joyce B. Main,et al.  Student evaluations of team members: Is there gender bias? , 2015, 2015 IEEE Frontiers in Education Conference (FIE).

[22]  Imad H. Elhajj,et al.  Turning Student Groups into Effective Teams , 2004 .

[23]  Jule B. Gassenheimer,et al.  Exploring the “Lone Wolf” Phenomenon in Student Teams , 2005 .

[24]  Gail E. Kaiser,et al.  Retina: helping students and instructors based on observed programming activities , 2009, SIGCSE '09.

[25]  Christoph Treude,et al.  Assessing developer contribution with repository mining-based metrics , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[26]  Alberto Sillitti,et al.  Cooperation, collaboration and pair-programming: Field studies on backup behavior , 2014, J. Syst. Softw..

[27]  Paola Spoletini,et al.  Measuring Team Members’ Contributions in Software Engineering Projects using Git-driven Technology , 2018, 2018 IEEE Frontiers in Education Conference (FIE).

[28]  Pei-Chi Chen,et al.  Effects of the Meetings-Flow Approach on Quality Teamwork in the Training of Software Capstone Projects , 2014, IEEE Transactions on Education.

[29]  Cheng-Chih Wu,et al.  Tracking Students’ Cognitive Processes During Program Debugging—An Eye-Movement Approach , 2016, IEEE Transactions on Education.

[30]  S. Thompson Social Learning Theory , 2008 .

[31]  Anson Seers,et al.  Team-member exchange quality: a new construct for role-making research , 1989 .

[32]  David Hovemeyer,et al.  Analyzing Student Work Patterns Using Programming Exercise Data , 2015, SIGCSE.

[33]  Gang Lu,et al.  Assessing Teamwork Performance in Software Engineering Education: A Case in a Software Engineering Undergraduate Course , 2011, 2011 18th Asia-Pacific Software Engineering Conference.

[34]  Louis Glassy,et al.  Using version control to observe student software development processes , 2006 .

[35]  Hye-Jung Lee,et al.  Peer Evaluation in Blended Team Project-Based Learning: What Do Students Find Important? , 2009, J. Educ. Technol. Soc..

[36]  Jesper Andersson,et al.  Good Practices for Educational Software Engineering Projects , 2007, 29th International Conference on Software Engineering (ICSE'07).

[37]  Jihie Kim,et al.  Assessing Collaborative Undergraduate Student Wikis and SVN with Technology-based Instrumentation: Relating Participation Patterns to Learning , 2011 .

[38]  Eleni Stroulia,et al.  Using CVS Historical Information to Understand How Students Develop Software , 2004, MSR.

[39]  Po-Yao Chao,et al.  Exploring students' computational practice, design and performance of problem-solving through a visual programming environment , 2016, Comput. Educ..