Does Transactional Memory Keep Its Promises? Results from an Empirical Study.

Transactional Memory (TM) promises to simplify parallel programming by replacing locks with atomic transactions. This is the first paper to assess the value proposition of TM based on a comparative case study with real programmers. Twelve students, working in teams of two, wrote a parallel desktop search engine in C/C++ during a fifteen week lab. Three randomly chosen study groups (TM teams) competed for the best performance using Intel’s Software Transactional Memory compiler and Pthreads, while three control groups (locks teams) competed using just Pthreads. The study provides empirical evidence that both supports the TM value proposition and at the same time points to problems with TM. The winning TM team’s program performed better than that of the winning locks team, and the TM winners were the first to have a prototype parallel search engine, four weeks earlier than the locks winners. Compared to the locks teams, the TM teams spent less than half the time debugging segmentation faults. On the other hand, TM teams had more problems tuning performance because TM performance was hard to predict. The study also provides insights into general difficulties programmers have with parallel programming. Some insights are technical; for example, some students wrongly assumed that it is safe to read shared variables outside critical sections, resulting in data races in the winning teams’ programs. Other insights are non-technical, including psychological ones; for example, the TM teams were less afraid of using parallel constructs, yet two of the teams procrastinated parallelization. Based on our insights, we elaborate on future research directions. We suggest refinements to TM constructs and tools. We also sketch how to automate similar case studies and propose a methodology for automating the evaluation of new language features for parallelism.