How type errors were fixed and what students did?

Providing better supports for debugging type errors has been an active research area in the last three decades. Numerous approaches from different perspectives have been developed. Most approaches work well under certain conditions only, for example, when type errors are caused by single leaves and when type annotations are correct. However, the research community is still unaware of which conditions hold in practice and what the real debugging situations look like. We address this problem with a study of 3 program data sets, which were written in different years, using different compilers, and were of diverse sizes. They include more than 55,000 programs, among which more than 2,700 are ill typed. We investigated all the ill-typed programs, and our results indicate that current error debugging support is far from sufficient in practice since only about 35% of all type errors were caused by single leaves. In addition, type annotations cannot always be trusted in error debuggers since about 30% of the time type errors were caused by wrong type annotations. Our study also provides many insights about the debugging behaviors of students in functional programming, which could be exploited for developing more effective error debuggers.

[1]  Kathi Fisler,et al.  Measuring the effectiveness of error messages designed for novice programmers , 2011, SIGCSE.

[2]  MANUEL M. T. CHAKRAVARTY,et al.  The risks and benefits of teaching purely functional programming in first year , 2004, Journal of Functional Programming.

[3]  Jonathan P. Munson,et al.  Analyzing novice programmers' response to compiler error messages , 2016 .

[4]  Martin Erwig,et al.  Guided Type Debugging , 2014, FLOPS.

[5]  Martin Erwig,et al.  Counter-factual typing for debugging type errors , 2014, POPL.

[6]  Jurriaan Hage,et al.  Neon: A Library for Language Usage Analysis , 2008, SLE.

[7]  Peter J. Stuckey,et al.  Improving type error diagnosis , 2004, Haskell '04.

[8]  Peyton Jones,et al.  Haskell 98 language and libraries : the revised report , 2003 .

[9]  Master Thesis,et al.  Analyzing Helium Programs Obtained Through Logging - The process of mining novice Haskell programs - , 2006 .

[10]  Peter J. Stuckey,et al.  Interactive type debugging in Haskell , 2003, Haskell '03.

[11]  Ewan D. Tempero,et al.  All syntax errors are not equal , 2012, ITiCSE '12.

[12]  Martin Erwig,et al.  Extending Type Inference to Variational Programs , 2014, ACM Trans. Program. Lang. Syst..

[13]  Baijun Wu,et al.  Learning user friendly type-error messages , 2017, Proc. ACM Program. Lang..

[14]  Olaf Chitil,et al.  Compositional explanation of types and algorithmic debugging of type errors , 2001, ICFP '01.

[15]  Bruce J. McAdam,et al.  Repairing type errors in functional programs , 2002 .

[16]  Danfeng Zhang,et al.  Toward general diagnosis of static errors , 2014, POPL.

[17]  Zvonimir Pavlinovic,et al.  Practical SMT-based type error localization , 2015, ICFP.

[18]  Ville Tirronen,et al.  Study on difficulties and misconceptions with modern type systems , 2014, ITiCSE '14.

[19]  Gregory John Michaelson,et al.  Improved Type Error Reporting , 2000 .

[20]  Jurriaan Hage,et al.  Heuristics for Type Error Discovery and Recovery , 2006, IFL.

[21]  Martin Erwig,et al.  Let's hear both sides: On combining type-error reporting tools , 2014, 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[22]  Kathi Fisler,et al.  Mind your language: on novices' interactions with error messages , 2011, Onward! 2011.

[23]  Thomas Schilling,et al.  Constraint-Free Type Error Slicing , 2011, Trends in Functional Programming.

[24]  Martin Erwig,et al.  An error-tolerant type system for variational lambda calculus , 2012, ICFP.

[25]  Phil Trinder,et al.  Explaining Polymorphic Types , 2002, Comput. J..

[26]  Christopher Scaffidi,et al.  The function, and dysfunction, of information sources in learning functional programming , 2012 .

[27]  Daan Leijen,et al.  Helium, for learning Haskell , 2003, Haskell '03.

[28]  James B. Fenwick,et al.  Another look at the behaviors of novice programmers , 2009, SIGCSE '09.

[29]  Christian Haack,et al.  Type error slicing in implicitly typed higher-order languages , 2003, Sci. Comput. Program..

[30]  Simon L. Peyton Jones,et al.  Diagnosing type errors with class , 2015, PLDI.

[31]  Dominic Duggan,et al.  Explaining Type Inference , 1996, Sci. Comput. Program..

[32]  Bastiaan Heeren,et al.  Top quality type error Messages , 2005 .

[33]  Jun Yang.  Explaining Type Errors by Finding the Source of a Type Conflict , 1999, Scottish Functional Programming Workshop.

[34]  Colin Myers,et al.  The Dys-Functional Student , 1995, FPLE.

[35]  Zvonimir Pavlinovic,et al.  Finding minimum type error sources , 2014, Software Engineering & Management.

[36]  Mitchell Wand.  Finding the source of type errors , 1986, POPL '86.

[37]  Olaf Chitil,et al.  Enumerating Counter-Factual Type Error Messages with an Existing Type Checker , 2014 .

[38]  Cesare Tinelli,et al.  Solving SAT and SAT Modulo Theories: From an abstract Davis--Putnam--Logemann--Loveland procedure to DPLL(T) , 2006, JACM.

[39]  P MunsonJonathan,et al.  Analyzing novice programmers' response to compiler error messages , 2016 .

[40]  Kwangkeun Yi,et al.  Proofs of a set of hybrid let-polymorphic type inference algorithms , 2009, New Generation Computing.

[41]  Stef Joosten,et al.  Teaching functional programming to first-year students , 1993, Journal of Functional Programming.

[42]  Manu Sridharan,et al.  Tech Report : A Practical Framework for Type Inference Error Explanation , 2016 .

[43]  Jurriaan Hage,et al.  Mining for Helium , 2006 .

[44]  Kwangkeun Yi,et al.  Proofs about a folklore let-polymorphic type inference algorithm , 1998, TOPL.

[45]  Bart Demoen,et al.  The computational power and complexity of constraint handling rules , 2009, TOPL.

[46]  Ville Tirronen,et al.  Understanding beginners' mistakes with Haskell , 2015, Journal of Functional Programming.

[47]  Kenichi Asai,et al.  An Embedded Type Debugger , 2012, IFL.

[48]  Kwangkeun Yi,et al.  A Generalized Let-Polymorphic Type Inference Algorithm , 2000 .

[49]  Dan Grossman,et al.  Seminal: searching for ML type-error messages , 2006, ML '06.

[50]  Dan Grossman,et al.  Searching for type-error messages , 2007, PLDI '07.