Threading is Sticky

The Guardian ---the fifth most widely read online newspaper in the world as of 2014---changed conversations on its commenting platform by altering its design from non-threaded to single-level threaded in 2012. We studied this naturally occurring experiment to investigate the impact of conversation threading on user retention as mediated by several potential changes in conversation structure and style. Our analysis shows that the design change made new users significantly more likely to comment a second time, and that this increased stickiness is due in part to a higher fraction of comments receiving responses after the design change. In mediation analysis, other anticipated mechanisms such as reciprocal exchanges and comment civility did not help to explain users' decision to return to the commenting system; indeed, civility did not increase after the design change and reciprocity declined. These analyses show that even simple design choices can have a significant impact on news forums' stickiness. Further, they suggest that this influence is more powerfully shaped by affordances---the new system made responding easier---than by changes in users' attention to social norms of reciprocity or civility. This has an array of implications for designers.