A data warehouse/online analytic processing framework for web usage mining and business intelligence reporting

Web usage mining is the application of data mining techniques to discover usage patterns and behaviors from web data (clickstream, purchase information, customer information, etc.) in order to understand and serve e‐commerce customers better and improve the online business. In this article, we present a general data warehouse/online analytic processing (OLAP) framework for web usage mining and business intelligence reporting. When we integrate the web data warehouse construction, data mining, and OLAP into the e‐commerce system, this tight integration dramatically reduces the time and effort for web usage mining, business intelligence reporting, and mining deployment. Our data warehouse/OLAP framework consists of four phases: data capture, webhouse construction (clickstream marts), pattern discovery and cube construction, and pattern evaluation and deployment. We discuss data transformation operations for web usage mining and business reporting in clickstream, session, and customer levels; describe the problems and challenging issues in each phase in detail; provide plausible solutions to the issues; and demonstrate the framework with some examples from some real web sites. Our data warehouse/OLAP framework has been integrated into some commercial e‐commerce systems. We believe this data warehouse/OLAP framework would be very useful for developing any real‐world web usage mining and business intelligence reporting systems. © 2004 Wiley Periodicals, Inc.

[1]  Ron Kohavi,et al.  Mining e-commerce data: the good, the bad, and the ugly , 2001, KDD '01.

[2]  Ron Kohavi,et al.  Integrating e-commerce and data mining: architecture and challenges , 2000, Proceedings 2001 IEEE International Conference on Data Mining.

[3]  Lars Schmidt-Thieme,et al.  Mining Web Navigation Path Fragments , 2002 .

[4]  C. Theusinger,et al.  Analyzing the footsteps of your customers , 2000 .

[5]  Geoff Hulten,et al.  A General Method for Scaling Up Machine Learning Algorithms and its Application to Clustering , 2001, ICML.

[6]  Michael J. A. Berry,et al.  Mastering Data Mining: The Art and Science of Customer Relationship Management , 1999 .

[7]  Il-Yeol Song,et al.  Data Warehouse Design for E-Commerce Environments , 1999, ER.

[8]  Jaideep Srivastava,et al.  Web usage mining: discovery and applications of usage patterns from Web data , 2000, SKDD.

[9]  Il-Yeol Song,et al.  Database Design for Real-World E-Commerce Systems. , 2000 .

[10]  Ralph Kimball,et al.  The Data Webhouse Toolkit: Building the Web-enabled Data Warehouse , 2000, Ind. Manag. Data Syst..

[11]  Dennis Murray,et al.  Data warehousing in the real world - a practical guide for building decision support systems , 1997 .

[12]  Pang-Ning Tan,et al.  Modeling of Web Robot Navigational Patterns , 2000 .

[13]  Hendrik Blockeel,et al.  Web mining research: a survey , 2000, SKDD.

[14]  James E. Pitkow,et al.  Characterizing Browsing Behaviors on the World-Wide Web , 1995 .