Uncovering unintended and shadow practices of users of decision support system dashboards in higher education institutions

Higher education institutions' (HEI) have begun to develop decision support system data dashboards (DSS‐DD) to improve the data‐informed decision making practices of institutional decision makers. This qualitative study examines the practices of decision makers as they engage with DSS‐DD at a large U.S. Midwestern university and uncovers the socio‐technical characteristics that lead to limited or non‐use of dashboards. To examine these practices and characteristics, this study presents a framework grounded in socio‐technical interaction networks from social informatics and sociomateriality from information systems that explores the socio‐technical practices of users within organizations, while acknowledging the impact of the users' socio‐technical contexts on their DSS‐DD practices. The results show that during the design and implementation phases of these dashboards the institutional contexts that the dashboards are meant to inform are often ignored; and that as users interact with these systems they develop unintended and shadow practices that lead to limited or non‐use of the dashboards for decision making purposes. Additionally, the study finds that users' practices are influenced by their local socio‐technical networks, which includes their prior experiences using institutional data, other actors within their institutional unit, and the political and social contexts which shape the users' decision making behavior and data‐use practices.

[1]  Sue Newell,et al.  Understanding Project Survival in an ES Environment: A Sociomaterial Practice Perspective , 2010, J. Assoc. Inf. Syst..

[2]  Hedley Rees Information Systems and Information Technology , 2011 .

[3]  Christopher Marquis,et al.  Imprinting: Toward A Multilevel Theory , 2013 .

[4]  D. Teece Profiting from technological innovation: Implications for integration, collaboration, licensing and public policy , 1993 .

[5]  K. Webber,et al.  Institutional Research in South African Higher Education: Framing the Contexts and Practices , 2016 .

[6]  Teofilo F. Gonzalez,et al.  Computing Handbook, Third Edition , 2014 .

[7]  Joshua J. Mitchell,et al.  Developing and Using Dashboard Indicators in Student Affairs Assessment , 2013 .

[8]  Howard Rosenbaum,et al.  Socio-Technical interaction networks as a tool for understanding digital libraries , 2004, ASIST.

[9]  S. Sawyer Social informatics: Overview, principles and opportunities , 2006 .

[10]  S Finkelstein,et al.  Power in top management teams: dimensions, measurement, and validation. , 1992, Academy of Management journal. Academy of Management.

[11]  Rob Kling,et al.  Reconceptualizing Users as Social Actors in Information Systems Research , 2003, MIS Q..

[12]  Karen Ruhleder,et al.  Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces , 1996, Inf. Syst. Res..

[13]  Fiorella Foscarini,et al.  Diplomatics and genre theory as complementary approaches , 2012 .

[14]  W. Orlikowski The sociomateriality of organisational life: considering technology in management research , 2010 .

[15]  Stefano Breschi,et al.  Tracing the links between science and technology: An exploratory analysis of scientists' and inventors' networks , 2010 .

[16]  Pnina Fichman,et al.  Social Informatics Evolving , 2015, Synthesis Lectures on Information Concepts, Retrieval, and Services.

[17]  Jennie Carroll,et al.  Excluded again: implications of integrated e-government systems for those at the margins , 2008, Inf. Technol. People.

[18]  D. Hambrick,et al.  Upper Echelons: The Organization as a Reflection of Its Top Managers , 1984 .

[19]  Paul Dourish,et al.  Dynamic Reconfiguration in Planetary Exploration: A Sociomaterial Ethnography , 2014, MIS Q..

[20]  Michael Roach,et al.  A Taste for Science? PhD Scientists’ Academic Orientation and Self-Selection into Research Careers in Industry , 2010 .

[21]  Eric T. Meyer,et al.  Creation and Control in the Photographic Process: iPhones and the emerging fifth moment of photography , 2012 .

[22]  J. Hicks,et al.  The economics of science , 1996 .

[23]  Berkant Barla Cambazoglu,et al.  Scalability Challenges in Web Search Engines , 2015, Scalability Challenges in Web Search Engines.

[24]  Rob Kling,et al.  a Bit More to It: Scholarly Communication Forums as Socio-technical Interaction Networks , 2003, J. Assoc. Inf. Sci. Technol..

[25]  Michael F. Middaugh Communicating Assessment Results , 2012 .

[26]  Dubravka Cecez-Kecmanovic,et al.  Reframing Success and Failure of Information Systems: A Performative Perspective , 2014, MIS Q..

[27]  D. Hicks Published Papers, Tacit Competencies and Corporate Management of the Public/Private Character of Knowledge , 1995 .

[28]  Satomi Sugiyama,et al.  Kawaii meiru and Maroyaka neko: Mobile emoji for relationship maintenance and aesthetic expressions among Japanese teens , 2015, First Monday.

[29]  Charles O. Kile,et al.  Using Industry Classification Codes to Sample High-Technology Firms: Analysis and Recommendations , 2009 .

[30]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[31]  Rob Kling,et al.  What Is Social Informatics and Why Does It Matter? , 2007, D Lib Mag..

[32]  Margaret A. Peteraf The cornerstones of competitive advantage: A resource‐based view , 1993 .

[33]  Ben Williamson,et al.  Digital education governance: data visualization, predictive analytics, and ‘real-time’ policy instruments , 2016 .

[34]  Ben Daniel,et al.  Big Data and analytics in higher education: Opportunities and challenges , 2015, Br. J. Educ. Technol..

[35]  Sue Newell,et al.  The Sociomaterialty of Information Systems: Current Status, Future Directions , 2014, MIS Q..

[36]  Margaret Cohen,et al.  Institutional Dashboards: Navigational Tool for Colleges and Universities. Professional File. Number 123, Winter 2012. , 2012 .

[37]  Rom Harré,et al.  Material Objects in Social Worlds , 2002 .

[38]  Susan V. Scott,et al.  Exploring Material‐Discursive Practices , 2015 .

[39]  S. Frickel,et al.  A General Theory of Scientific/Intellectual Movements , 2005 .

[40]  N. Warner,et al.  VOWELS AND NAIL POLISH : THE EMERGENCE OF LINGUISTIC STYLE IN THE PREADOLESCENT HETEROSEXUAL MARKETPLACE , 2002 .

[41]  K. Baker,et al.  Toward Information Infrastructure Studies: Ways of Knowing in a Networked Environment , 2009 .

[42]  Steve Sawyer,et al.  The Sociotechnical Nature of Mobile Computing Work: Evidence from a Study of Policing in the United State , 2005, Int. J. Technol. Hum. Interact..

[43]  Benoît Godin,et al.  Research and the practice of publication in industries , 1996 .

[44]  Eric T. Meyer Socio-Technical Interaction Networks: A Discussion of the Strengths, Weaknesses and Future of Kling's STIN Model , 2006, HCC.

[45]  Eric T. Meyer,et al.  Examining the Hyphen: The Value of Social Informatics for Research and Teaching , 2014 .

[46]  Steve Sawyer,et al.  Social Informatics: Principles, Theory, and Practice , 2006, HCC.

[47]  Andrew Abbott,et al.  Linked Ecologies: States and Universities as Environments for Professions* , 2005 .

[48]  Craig Gibson,et al.  Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education , 2000 .

[49]  Steve Sawyer,et al.  A Social Informatics Perspective on Socio-Technical Networks , 2000 .

[50]  Chase McCoy,et al.  Addressing the Technical in Socio-Technical Systems Research: A Review of Social Informatics Literature , 2017, AMCIS.

[51]  G. Steven McMillan,et al.  Firm management of scientific information: an empirical update , 2000 .

[52]  Mohammad Hossein Jarrahi,et al.  Examining Practice, Structural, and Interpretive Dimensions of Technology Assemblages: The Case of Social Technology Use by Knowledge Workers , 2014, ASIST.

[53]  Susan V. Scott,et al.  10 Sociomateriality: Challenging the Separation of Technology, Work and Organization , 2008 .

[54]  Leah P. Macfadyen,et al.  Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. , 2014 .

[55]  Jay Liebowitz,et al.  Thoughts on Recent Trends and Future Research Perspectives in Big Data and Analytics in Higher Education , 2017 .

[56]  Matthew Jones,et al.  A Matter of Life and Death: Exploring Conceptualizations of Sociomateriality in the Context of Critical Care , 2014, MIS Q..

[57]  Christian Sandvig,et al.  Infrastructure studies meet platform studies in the age of Google and Facebook , 2018, New Media Soc..

[58]  Ya-Ching Lee,et al.  Effects of branded e-stickers on purchase intentions: The perspective of social capital theory , 2017, Telematics Informatics.

[59]  Ben Kei Daniel,et al.  Technology Enhanced Analytics (TEA) in Higher Education. , 2013 .

[60]  CHRISTINE L. BORGMAN,et al.  Digital libraries and the continuum of scholarly communication , 2000, J. Documentation.

[61]  Paul Prinsloo Some Provocations for a Future of Institutional Research: Evidence-Based Decision Making as Séance , 2016 .

[62]  Howard Rosenbaum,et al.  AIS Electronic Library (AISeL) , 2006 .

[63]  Karin Garrety,et al.  Doing Infrastructural Work: The Role of Boundary Objects in Health Information Infrastructure Projects , 2016, Scand. J. Inf. Syst..

[64]  Rob Kling,et al.  Learning About Information Technologies and Social Change: The Contribution of Social Informatics , 2000, Inf. Soc..

[65]  Jennie Carroll,et al.  Towards Anticipating IS Consequences: An Anatomy of Sociotechnical Interaction Networks (STINs) , 2008 .

[66]  Mohammad Hossein Jarrahi,et al.  Mobility of knowledge work and affordances of digital technologies , 2017, Int. J. Inf. Manag..

[67]  Mohammad Hossein Jarrahi,et al.  Agency, sociomateriality, and configuration work , 2018, Inf. Soc..

[68]  Emily A. Marshall Defining population problems: Using topic models for cross-national comparison of disciplinary development , 2013 .

[69]  Alan Bain,et al.  Transforming the Measurement of Learning and Teaching in Higher Education , 2016 .

[70]  Lotta Hultin,et al.  Visualizing institutional logics in sociomaterial practices , 2014, Inf. Organ..

[71]  Johan P. Olsen Accountability and Ambiguity , 2014 .

[72]  Rob Kling,et al.  The Web of Computing: Computer Technology as Social Organization , 1982, Adv. Comput..

[73]  Wanda J. Orlikowski,et al.  Entanglements in Practice: Performing Anonymity Through Social Media , 2014, MIS Q..

[74]  Markus Simeth,et al.  How do firms develop capabilities for scientific disclosure , 2015 .