An infrastructure for workplace learning analytics: Tracing knowledge creation with the social semantic server

In this paper, we propose the Social Semantic Server (SSS) as a service-based infrastructure for workplace and professional learning analytics (LA). The design and development of the SSS have evolved over eight years, starting with an analysis of workplace learning inspired by knowledge creation theories and their application in different contexts. The SSS collects data from workplace learning tools, integrates it into a common data model based on a semantically enriched artifact-actor network, and offers it back for LA applications to exploit the data. Further, the SSS design’s flexibility enables it to be adapted to different workplace learning situations. This paper contributes by systematically deriving requirements for the SSS according to knowledge creation theories, and by offering support across a number of different learning tools and LA applications integrated into the SSS. We also show evidence for the usefulness of the SSS extracted from 4 authentic workplace learning situations involving 57 participants. The evaluation results indicate that the SSS satisfactorily supports decision making in diverse workplace learning situations and allow us to reflect on the importance of knowledge creation theories for this analysis.

[1]  Ulrik Schroeder,et al.  Evolution of Interests in the Learning Context Data Model , 2015, EC-TEL.

[2]  Dragan Gasevic,et al.  Recipe for success: lessons learnt from using xAPI within the connected learning analytics toolkit , 2016, LAK.

[3]  Marko Becker,et al.  Service Oriented Architecture Concepts Technology And Design , 2016 .

[4]  Katja Niemann,et al.  Usage-Based Clustering of Learning Resources to Improve Recommendations , 2014, EC-TEL.

[5]  E. Duval.  Attention please!: learning analytics for visualization and recommendation , 2011, LAK.

[6]  Tobias Ley,et al.  Making Sense of Bits and Pieces: A Sensemaking Tool for Informal Workplace Learning , 2014, EC-TEL.

[7]  Dominik Kowald,et al.  The social semantic server: a flexible framework to support informal learning at the workplace , 2015, I-KNOW.

[8]  John Cook,et al.  Going beyond your Personal Learning Network, Using Recommendations and Trust through a Multimedia Question-Answering Service for Decision-support: a Case Study in the Healthcare , 2016, J. Univers. Comput. Sci..

[9]  María Jesús Rodríguez-Triana,et al.  Learning Analytics for Professional and Workplace Learning: A Literature Review , 2017, IEEE Transactions on Learning Technologies.

[10]  Peter B. Sloep,et al.  Recommending peers for learning: Matching on dissimilarity in interpretations to provoke breakdown , 2017, Br. J. Educ. Technol..

[11]  Dominik Kowald,et al.  Long time no see: the probability of reusing tags as a function of frequency and recency , 2013, WWW.

[12]  Ralf Klamma,et al.  Community Learning Analytics - Challenges and Opportunities , 2013, ICWL.

[13]  Tobias Ley,et al.  Dynamics of human categorization in a collaborative tagging system: How social processes of semantic stabilization shape individual sensemaking , 2015, Comput. Hum. Behav..

[14]  Dietrich Albert,et al.  A Mashup Recommender for Creating Personal Learning Environments , 2012, ICWL.

[15]  K. Hakkarainen,et al.  The Knowledge Creation Metaphor – An Emergent Epistemological Approach to Learning , 2005 .

[16]  Tobias Ley,et al.  How Do People Learn at the Workplace? Investigating Four Workplace Learning Assumptions , 2007, EC-TEL.

[17]  Patricia Santos,et al.  KnowBrain: An Online Social Knowledge Repository for Informal Workplace Learning , 2015, EC-TEL.

[18]  Francisco J. García-Peñalvo,et al.  Using Learning Analytics to improve teamwork assessment , 2015, Comput. Hum. Behav..

[19]  Carlos Alario Hoyos.  GLUE!: an architecture for the integration of external tools in virtual learning environments , 2012 .

[20]  Ronald Maier,et al.  Technological and Organizational Arrangements Sparking Effects on Individual, Community and Organizational Learning , 2012, EC-TEL.

[21]  Katja Niemann,et al.  Contextualized attention metadata , 2011 .

[22]  Marek Hatala,et al.  Associations between technological scaffolding and micro-level processes of self-regulated learning: A workplace study , 2016, Comput. Hum. Behav..

[23]  Kalina Yacef,et al.  Analysis of collaborative writing processes using revision maps and probabilistic topic models , 2013, LAK '13.

[24]  Marek Hatala,et al.  Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning , 2016, Comput. Hum. Behav..

[25]  Tobias Ley,et al.  Supporting Learning Analytics for Informal Workplace Learning with a Social Semantic Infrastructure , 2015, EC-TEL.

[26]  Daniel D. Suthers,et al.  Multilevel Analysis of Uptake, Sessions, and Key Actors in a Socio-Technical Network , 2014, LAK Workshops.

[27]  Ronald Maier,et al.  Conceptual Foundations for a Service-oriented Knowledge and Learning Architecture: Supporting Content, Process and Ontology Maturing , 2009, Networked Knowledge - Networked Media - Integrating Knowledge Management.

[28]  Dominik Kowald,et al.  Refining Frequency-Based Tag Reuse Predictions by Means of Time and Semantic Context , 2013, MSM/MUSE.

[29]  Paul Seitlinger,et al.  Take up My Tags: Exploring Benefits of Meaning Making in a Collaborative Learning Task at the Workplace , 2016, EC-TEL.

[30]  M. Eraut,et al.  Informal learning in the workplace , 2004 .

[31]  Dominik Kowald,et al.  Recommending tags with a model of human categorization , 2013, CIKM.

[32]  Michael Derntl,et al.  A Dynamic Topic Model of Learning Analytics Research , 2013, LAK.

[33]  Marek Hatala,et al.  Learn-B: a social analytics-enabled tool for self-regulated workplace learning , 2012, LAK '12.

[34]  Markus F. Peschl,et al.  Designing and Enabling Spaces for collaborative knowledge creation and innovation: From managing to enabling innovation as socio-epistemological technology , 2014, Comput. Hum. Behav..

[35]  John Cook,et al.  Scaling informal learning at the workplace: A model and four designs from a large-scale design-based research effort , 2014, Br. J. Educ. Technol..

[36]  I. Nonaka.  A Dynamic Theory of Organizational Knowledge Creation , 1994 .

[37]  Tobias Ley,et al.  Visualizing Workplace Learning Data with the SSS Dashboard , 2016, CrossLAK.

[38]  Dominik Kowald,et al.  Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpretation Dynamics , 2015, WWW.

[39]  Dominik Kowald,et al.  Forgetting the Words but Remembering the Meaning: Modeling Forgetting in a Verbal and Semantic Tag Recommender , 2014, MSM/MUSE.

[40]  Prasun Dewan,et al.  An Integrated Approach to Designing and Evaluating Collaborative Applications and Infrastructures , 2004, Computer Supported Cooperative Work (CSCW).

[41]  Bieke Schreurs,et al.  Visualizing Informal Professional Development Networks , 2013 .

[42]  Tobias Ley,et al.  Which User Interactions Predict Levels of Expertise in Work-Integrated Learning? , 2013, EC-TEL.

[43]  Paul Seitlinger,et al.  Towards a user model for personalized recommendations in work-integrated learning: A report on an experimental study with a collaborative tagging system , 2010, RecSysTEL@RecSys.

[44]  Ralf Klamma,et al.  Supporting Self-Regulation by Personal Learning Environments , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[45]  Mehrbakhsh Nilashi,et al.  Collaborative filtering recommender systems , 2013 .

[46]  Dominik Kowald,et al.  ScaR: Towards a Real-Time Recommender Framework Following the Microservices Architecture , 2015, RecSys 2015.

[47]  Dominik Kowald,et al.  Improving Collaborative Filtering Using a Cognitive Model of Human Category Learning , 2017, J. Web Sci..

[48]  Äli Leijen,et al.  Perspectives for Defining Student Teacher Performance-Based Teaching Skills Indicators to Provide Formative Feedback through Learning Analytics , 2015 .

[49]  Mandy Lupton,et al.  Designing for student-facing learning analytics , 2017 .

[50]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[51]  Christine Kunzmann,et al.  Living documents as a collaboration and knowledge maturing platform , 2014, i-KNOW '14.

[52]  Sam Newman,et al.  Building microservices - designing fine-grained systems, 1st Edition , 2015 .

[53]  Ralf Klamma,et al.  From micro to macro: analyzing activity in the ROLE Sandbox , 2013, LAK '13.

[54]  Bettina Berendt,et al.  Learning Analytics and their Application in Technology-enhanced Professional Learning , 2013 .

[55]  Dan Frankowski,et al.  Collaborative Filtering Recommender Systems , 2007, The Adaptive Web.

[56]  Tobias Ley,et al.  Integrating Data Across Workplace Learning Applications with a Social Semantic Infrastructure , 2015, ICWL.

[57]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[58]  Dominik Kowald,et al.  Balancing the Fluency-Consistency Tradeoff in Collaborative Information Search with a Recommender Approach , 2018, Int. J. Hum. Comput. Interact..

[59]  Dominik Kowald,et al.  The TagRec Framework as a Toolkit for the Development of Tag-Based Recommender Systems , 2017, UMAP.

[60]  Dominik Kowald,et al.  The Influence of Frequency, Recency and Semantic Context on the Reuse of Tags in Social Tagging Systems , 2016, HT.

[61]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[62]  Claire Bradley,et al.  Designing social media for informal learning and knowledge maturing in the digital workplace , 2012, J. Comput. Assist. Learn..