An empirical evaluation of design rationale documents

While several studies propose methods and notations for "capturing" design rationale (DR), there is to date little data available on how useful this information is when a designer needs to reuse a previous design. This paper presents the results of an empirical evaluation of DR documents, carried out with six experienced professional designers who were asked to understand and to assess a past design. These designers were provided with documents that described the solution and documents describing the DR. These DR documents were constructed using the QOC method. To determine the usefulness of DR documents, we attempt to answer the three following questions: (1) Do designers confronted with an unknown design need to know the design rationales? (2) How designers use design rationale documents? (3) Do we succeed in capturing the rationales looked for by designers? The results provided by this study lead us to conclude that DR should be useful, at least for some designers who use it as a support to their reasoning, but not sufficient. Indeed, this study exhibits some limitations of the traditional approaches for recording DR. We discuss these limitations and some solutions needed to go beyond them.