Continuous archival and analysis of user data in virtual and immersive game environments

We present a continuous and unobtrusive approach to analyze and reason about users' personal experiences of interacting with virtual and game environments. Focusing on an immersive educational game environment that we are developing, this is achieved through the capture and storage of user's movements and events that occur as a result of interactions with and within immersive environments. Termed immersidata, we then query and analyze immersidata to make sense of user behavior.Two example approaches are described. The first describes an application ISIS (Immersidata analySIS) that provides a tool for analysis of user behavior/experience through the indexing of immersidata with video clips of students' gaming sessions. This approach is described by way of an example to identify the causes of interruptions or breaks in interactions/focus of attention to facilitate the identification of problematic design. In our second example we describe our work towards classifying students' performance through immersidata. To this aim, we describe one example of transforming immersidata into multivariate time series and then by applying feature subset selection techniques we identify the features that differentiate students. We describe the application of this approach to identify novice and expert players with 90\% accuracy. One proposal is to use this to customize the game environment appropriate to the students' ability. Finally, we present future directions for the continuation of the work presented herein and also, the application of the immersidata system to capture, store and analyze personal behavior/experiences and provide appropriate feedback in our work and home environments.

[1]  Mel Slater,et al.  A Virtual Presence Counter , 2000, Presence: Teleoperators & Virtual Environments.

[2]  Cyrus Shahabi,et al.  AIMS: An Immersidata Management System , 2003, CIDR.

[3]  Kiyoharu Aizawa,et al.  Efficient retrieval of life log based on context and content , 2004, CARPE'04.

[4]  Cyrus Shahabi,et al.  Automating the detection of breaks in continuous user experience with computer games , 2005, CHI Extended Abstracts.

[5]  Tsukasa Noma,et al.  PWS & PHA: posture web server and posture history archiver , 2004, CARPE'04.

[6]  Simon Brown,et al.  Affective gaming: measuring emotion through the gamepad , 2003, CHI Extended Abstracts.

[7]  Cyrus Shahabi,et al.  Immersidata Management: Challenges in Management of Data Generated within an Immersive Environment , 1999, Multimedia Information Systems.

[8]  C. Shahabi,et al.  A Supervised Feature Subset Selection Technique for Multivariate Time Series , 2005 .

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  Mel Slater,et al.  Physiological Responses to Breaks in Presence: A Pilot Study , 2003 .

[11]  Cyrus Shahabi,et al.  User Experiences and Lessons Learned from Developing and Implementing an Immersive Game for the Science Classroom , 2005 .

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  Cyrus Shahabi,et al.  Feature subset selection and feature ranking for multivariate time series , 2005, IEEE Transactions on Knowledge and Data Engineering.