The Value of Web Log Data in Use-Based Design and Testing

Web-based logs contain potentially useful empirical data with which World Wide Web (Web) designers and design theorists can assess usability and effectiveness of design choices. Most Web design guidelines from artistic or usability principles feature no empirical validation, while empirical studies of Web use typically rely on observer ratings. Web server logs and client-side logs can provide naturally-occurring, unobtrusive usage data, partially amenable to normative use assessments but particularly useful in experimental research comparing alternative Web designs. Identification of types of Web server logs, client logs, types and uses of log data, and issues associated with the validity of these data are enumerated. Frameworks that outline how sources of use-based data can be triangulated to assess Web design are illustrated. Finally, an approach to experimentation that overcomes many data validity issues is presented and illustrated through a pilot experiment that used server logs to compare user responses to frames, pop-up, and scrolling arrangements of a single Web site.