Content or Graphics? An Empirical Analysis of Criteria for Award-Winning Websites

What makes an award-winning website? Is content more important than graphics, or is a splashy visual design the determining factor for acquiring accolades? To provide an empirical basis for answers to these and related questions, we examined the Webby Awards 2000 dataset to understand which factors distinguish highly-rated websites from those that receive poor ratings. For these awards, the websites were categorized into 27 topical categories such as Science, Arts, Commerce, Living, and News, and expert judges were recruited for each of these topic areas. The websites, numbering nearly 3000, were rated according to six criteria: content, structure & navigation, visual design, functionality, interactivity, and overall experience. We found that, across the dataset, the content criterion was by far the best predictor of the overall experience criterion, while the visual design criterion was the worst predictor of the overall experience. Analysis of ratings within the 27 topical categories showed that the importance of the five criteria to the overall experience score differed substantially among the topics. For example, in the Arts category, the visual design criterion and the content criterion were equally important, whereas in the Activism category, content was very important, but visual design was not important for predicting the score for overall experience. The importance of content and the relative lack of importance of visual design was mirrored in layperson’s ratings of the websites (known as the People’s Voice ratings). Websites with the highest ratings as determined by peoples' votes were those that rated lower in visual design and higher in content by the Webby Awards judges.