Toward preprototype user acceptance testing of new information systems: implications for software project management

Errors in requirements specifications have been identified as a major contributor to costly software project failures. It would be highly beneficial if information systems developers could verify requirements by predicting workplace acceptance of a new system based on user evaluations of its specifications measured during the earliest stages of the development project, ideally before building a working prototype. However, conventional wisdom among system developers asserts that prospective users must have direct hands-on experience with at least a working prototype of a new system before they can provide assessments that accurately reflect future usage behavior after workplace implementation. The present research demonstrates that this assumption is only partially true. Specifically, it is true that stable and predictive assessments of a system's perceived ease of use should be based on direct behavioral experience using the system. However, stable and behaviorally predictive measures of perceived usefulness can be captured from target users who have received information about a system's functionality, but have not had direct hands-on usage experience. This distinction is key because, compared to ease of use, usefulness is generally much more strongly linked to future usage intentions and behaviors in the workplace. Two longitudinal field experiments show that preprototype usefulness measures can closely approximate hands-on based usefulness measures, and are significantly predictive of usage intentions and behavior up to six months after workplace implementation. The present findings open the door toward research on how user acceptance testing may be done much earlier in the system development process than has traditionally been the case. Such preprototype user acceptance tests have greater informational value than their postprototype counterparts because they are captured when only a relatively small proportion of project costs have been incurred and there is greater flexibility to modify a new system's design attributes. Implications are discussed for future research to confirm the robustness of the present findings and to better understand the practical potential and limitations of preprototype user acceptance testing.