Chapter 58: Analysis of Log File Data to Understand Behavior and Learning in an Online Community

How do we study the behavior of users in online communities? Researchers from a variety of disciplines have evolved a rich set of both quantitative and qualitative approaches to studying human‐computer interaction (HCI) and computer-mediated communication (CMC), and these methods are useful in the study of online communities. Unique to the study of CMC and online communities is the possibility of collecting log file data .I ti spossible for the computer to record every command typed by users—in some cases, every keystroke. In cases where users interact only online, we can collect a comprehensive record of all of their interactions. The completeness of the record and ease of collecting it are unprecedented. However, log file data is more often collected than analyzed. We can save everything, but what does it mean? This chapter presents two examples of the use of log file data to understand user behavior and learning in one online environment, MOOSE Crossing. MOOSE Crossing is a text-based virtual reality environment (or “MUD”) in which kids aged eight and older learn objectoriented programming and practice their creative writing. From analysis of log file data from the environment, we gained significant new insights into user behavior and learning. We will briefly discuss an example of qualitative log file analysis, in which a close reading of records of online interaction provides insights into how children learn from one another in this environment (Bruckman, 2000). The rest of the chapter contains an extended example of the use of quantitative log file analysis to try to understand whether there is any relationship between gender and programming achievement on this site. (An earlier version of this analysis was published in Proceedings of CSCL 2002 (Bruckman et al., 2002).) Quantitative and qualitative methods are generally more powerful when used in a complementary fashion. In the data to follow, our qualitative findings suggested that girls are especially strong achievers in this learning environment. However, quantitative analysis of log file data shows that boys and girls use the environment in similar ways. A small group of girls form the core of “the regulars”, the most active, dedicated members who are highly visible to both researchers and other members. The regulars have extraordinary levels