Programming Skill, Knowledge and Working Memory Among Software Developers from an Investment Theory Perspective

This study investigates the role of working memory and experience in the development of programming knowledge and programming skill. An instrument for assessing programming skill where skill is inferred from programming performance was administered along with tests of working memory and programming knowledge. We hired 65 professional software developers from nine companies in eight European countries to participate in a two-day study. Results indicate that the effect of working memory and experience on programming skill is mediated through programming knowledge. Programming knowledge was further found to explain individual differences in programming skill to a large extent. The overall findings support Cattell’s investment theory. This work contributes to research on individual differences in semirealistic work settings where professionals are used as subjects. This is a preprint version of: Bergersen, G. R., & Gustafsson, J.-E. (2011). Programming skill, knowledge and working memory among professional software developers from an investment theory perspective. Journal of Individual Differences 32(4), 201-209. doi: 10.1027/1614-0001/a000052. Accepted 2011-Jan-10 Not suitable for citation Copyright Hogrefe Publishing 2011 PROGRAMMING SKILL, KNOWLEDGE AND WORKING MEMORY 3 PROGRAMMING SKILL, KNOWLEDGE AND WORKING MEMORY AMONG SOFTWARE DEVELOPERS FROM AN INVESTMENT THEORY PERSPECTIVE Software systems are a cornerstone of modern society. Software production is a highly competitive and globalized industry that is focused on producing high quality software at low cost. One important way to stay competitive is to recruit and retain highly productive software developers. Although companies often use different methods for quantifying competence when recruiting developers, tests of cognitive abilities are frequently employed. Generally, cognitive abilities can be organized into a hierarchical structure (Gustafsson, 2002). At the apex, general mental ability (g) exerts influence over all lower factors. At the next stratum are a handful of broad abilities such as crystallized g (Gc, acquired knowledge) and fluid g (Gf, novel and abstract problem solving), while the lowest stratum specifies a large number of narrow abilities. Gustafsson (1984) demonstrated that Gf and g are perfectly correlated, and Valentin Kvist and Gustafsson (2008) showed that this perfect relation holds true only for homogeneous populations in which the individuals have had reasonably similar opportunities to acquire the knowledge and skills tested. According to Cattell’s investment theory (1971/1987), the acquisition of knowledge and skills and the formation of developed abilities (i.e., Gc) are influenced by Gf, effort, motivation and interest, and also by previous levels of Gc (Valentin Kvist and Gustafsson, 2008). Because Gf is involved in all new learning, this ability is identical with g. Gf (and g) also has a wider breadth of influence than other factors of intelligence, but it does not necessarily exert a strong influence on performance on any single task (Humphreys, 1962; Coan, 1964; Gustafsson, 2002; Valentin Kvist & Gustafsson, 2008). This is a preprint version of: Bergersen, G. R., & Gustafsson, J.-E. (2011). Programming skill, knowledge and working memory among professional software developers from an investment theory perspective. Journal of Individual Differences 32(4), 201-209. doi: 10.1027/1614-0001/a000052. Accepted 2011-Jan-10 Not suitable for citation Copyright Hogrefe Publishing 2011 PROGRAMMING SKILL, KNOWLEDGE AND WORKING MEMORY 4 Based on the investment theory, as well as later extensions to this theory (see Ackerman, 2000), we expect that the influence of Gf and experience on skills as well as job performance to be mediated through knowledge. That a mediating relationship exists is supported by theories of skilled behaviour and skill acquisition (Neves & Anderson, 1981; Anderson, Conrad, & Corbett, 1989), in which knowledge is a key component. For example, according to the three phases of skill acquisition proposed by Fitts and Posner (1967), declarative knowledge is central to the first cognitive phase where an individual “tries to ‘understand’ the task and what it demands” (p. 11). Anderson (1987) also states that knowledge initially “comes in declarative form and is used by weak methods to generate solutions [which] ... form new productions ... [and a] key step is the knowledge compilation process, which produces the domain specific skill” (p. 187). Further, knowledge is also a central component of adult intelligence (Ackerman, 2000) and can therefore be an important factor in the acquisition of new knowledge and skills (Ackerman, 2007). The investment theory also is consonant with earlier reported results on job performance. For example, in a meta-analysis by Schmidt, Hunter & Outerbridge (1986), the strongest determinant of job sample performance was job knowledge (.74), much higher than the direct path from experience (.08) or general mental ability (.04). Further, Schmidt (2002) states that individuals with high general cognitive ability acquire more job knowledge and acquire it faster, which leads to “higher levels of job performance” (p. 201). Also, in another meta-analysis, experience and job performance were found to be positively correlated, even though this relationship diminished over time (McDaniel, Schmidt & Hunter, 1988). Constructs close to the apex of the hierarchical model can be described as having high referent generality, and constructs that are highly specific to a limited situation have low referent generality (see, e.g., Coan, 1964). For example, programming skill in a single programming This is a preprint version of: Bergersen, G. R., & Gustafsson, J.-E. (2011). Programming skill, knowledge and working memory among professional software developers from an investment theory perspective. Journal of Individual Differences 32(4), 201-209. doi: 10.1027/1614-0001/a000052. Accepted 2011-Jan-10 Not suitable for citation Copyright Hogrefe Publishing 2011 PROGRAMMING SKILL, KNOWLEDGE AND WORKING MEMORY 5 language can be regarded as a narrow construct with low referent generality. It is well suited for predicting the outcomes of an individual for a specific programming language, but there might be limited transfer of knowledge and skills to, for example, other kinds of programming languages. When assessing constructs with low referent generality, it is therefore important to also assess constructs with high referent generality (Gustafsson, 2002). Working memory is a construct that has a close relationship to Gf and g and can further be regarded as a construct with high referent generality. Although this relationship is complex (Ackerman, Beier, & Boyle, 2002), several researchers have reported a large degree of overlap between working memory and g (Ackerman et al., 2002; Colom, Rebollo, Palacious, JuanEspinosa, & Kyllonen, 2004; Unsworth, Heitz, Schrock, & Engle, 2005). Working memory also has a substantial overlap with perceptual speed (Kyllonen & Stephens, 1990; Ackerman et al., 2002). In addition to these overlaps, limitations of working memory, which are revealed in theories of skilled behaviour, are an important source of error in skilled performance (Anderson, 1987). We maintain, therefore, that working memory is a suitable high referent generality construct for the purpose of assessing programming skills both from a Gf perspective and from a procedural learning perspective (Kyllonen & Stephens, 1990). Computer programming is sometimes described as one of several archetypes of complex cognitive behaviour; overall, programming requires the programmer to have a high level of declarative knowledge as well as much practice to perform well. Working memory has previously been found to be a good predictor of programming skill acquisition (Shute, 1991), as has experience (Arisholm & Sjøberg, 2004). It has also been noted that performance on tasks operating under skill constraints can be good predictors of other tasks as well. For example, for programming in the LISP programming language, Anderson (1987) states that “the best predictor This is a preprint version of: Bergersen, G. R., & Gustafsson, J.-E. (2011). Programming skill, knowledge and working memory among professional software developers from an investment theory perspective. Journal of Individual Differences 32(4), 201-209. doi: 10.1027/1614-0001/a000052. Accepted 2011-Jan-10 Not suitable for citation Copyright Hogrefe Publishing 2011 PROGRAMMING SKILL, KNOWLEDGE AND WORKING MEMORY 6 of individual subject differences in errors on problems that involved one LISP concept was number of errors on other problems that involved different concepts” (p. 203). Further, amount of errors was also correlated with amount of programming experience (see Anderson et al., 1989, for further details). Skill as a latent construct is inferred from observed performance that varies in both time and accuracy (or quality) (Fitts & Posner, 1967; Anderson, 1987). Together with knowledge and motivation, skill is one of three direct antecedents of performance (Campbell, McCloy, Oppler, & Sager, 1993). Skill is sometimes also referred to as procedural knowledge, and its acquisition consists of three overlapping phases. During each phase, different abilities (as antecedents) are hypothesized to exert different levels of influence on the acquisition of skill (Ackerman, 1988): General mental ability is predominant during the first cognitive phase, perceptual speed during the second associative phase, and psychomotor ability during the final and autonomous phase. Kyllonen and Woltz (1989) refer to the first phase as the “knowledge acquisition phase,” while phase two and three are skill acquisition and skill refinement, respectively. Much research on the prediction of job performance has relied on a construct of general mental ability indexed by test batteries which measure a mixture of Gf