Collective agility, paradox and organizational improvisation: the development of a particle physics grid

This paper examines systems development in a global collaborative community of high‐energy physics and offers insights and implications for agile systems development in other large scale and distributed settings. The paper studies the ongoing construction of the UK's computing grid for particle physics (GridPP), a grid that is itself part of the world's largest grid, the Large Hadron Collider Computing Grid. We observe in this project a collective, agile and distributed performance through which the Grid is constructed. We express this through the concept of ‘collective agility’ which captures a large distributed performance rather than the more conventional sense of agility as small‐group and deliberate systems development practices. The collective agility of GridPP is analysed as a process of ‘enacted emergence’ expressed through the dynamics of six improvisation paradoxes.

[1]  Chris Barry.  Decision-Making and Information Systems Development - A Conceptual Framework , 2005, ECIS.

[2]  Eric Overby,et al.  Enterprise agility and the enabling role of information technology , 2006, Eur. J. Inf. Syst..

[3]  K. Weick.  Improvisation as a Mindset for Organizational Analysis , 1998 .

[4]  A. Kellerman,et al.  The Constitution of Society : Outline of the Theory of Structuration , 2015 .

[5]  Wanda J. Orlikowski,et al.  Improvising Organizational Transformation Over Time: A Situated Change Perspective , 1996, Inf. Syst. Res..

[6]  Jorgen P. Bansler,et al.  A reappraisal of structured analysis: design in an organizational context , 1993, TOIS.

[7]  Pekka Abrahamsson,et al.  ‘Lots done, more to do’: the current state of agile systems development research , 2009, Eur. J. Inf. Syst..

[8]  Christine Halverson,et al.  Considering an organization's memory , 1998, CSCW '98.

[9]  J. Alberto Espinosa,et al.  Ambidextrous coping strategies in globally distributed software development projects , 2006, CACM.

[10]  W BoehmBarry.  A Spiral Model of Software Development and Enhancement , 1988 .

[11]  P. Berliner.  Thinking in Jazz: The Infinite Art of Improvisation , 1995 .

[12]  Jan Pries-Heje,et al.  Business agility and diffusion of information technology , 2006, Eur. J. Inf. Syst..

[13]  K. Weick,et al.  Organizing and the Process of Sensemaking , 2005 .

[14]  Henry Mintzberg,et al.  Strategy Formation in an Adhocracy. , 1985 .

[15]  A. Strauss.  Basics Of Qualitative Research , 1992 .

[16]  Peng Xu,et al.  Can distributed software development be agile? , 2006, CACM.

[17]  MathiassenLars,et al.  Business agility and diffusion of information technology , 2006 .

[18]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[19]  O. Hanseth,et al.  From Control to Drift: The Dynamics of Corporate Information Infrastructures , 2000 .

[20]  Claudio U. Ciborra,et al.  A theory of information systems based on improvisation , 1999 .

[21]  Marianne Kivelä.  Dynamic capabilities in small software firms , 2007 .

[22]  A. Pickering.  The mangle of practice : time, agency, and science , 1997 .

[23]  Richard Baskerville,et al.  Amethodical systems development: the deferred meaning of systems development methods , 2000 .

[24]  Jorgen P. Bansler,et al.  Improvisation in Information Systems Development , 2004, Relevant Theory and Informed Practice.

[25]  Tore Dybå.  Improvisation in Small Software Organizations , 2000, IEEE Software.

[26]  M. Cunha,et al.  Towards a Theory of Organizational Improvisation: Looking Beyond the Jazz Metaphor , 2003 .

[27]  Anil Gaba,et al.  Coda : Creativity and Improvisation in Jazzand Organizations: Implications Fororganizational Learning , 1998 .

[28]  Karlheinz Kautz,et al.  Persistent problems and practices in information systems development , 2007, Inf. Syst. J..

[29]  Anne S. Miner,et al.  Organizational Improvisation and Learning: A Field Study , 2001 .

[30]  Jürgen Börstler,et al.  Software product line modeling made practical , 2006, CACM.

[31]  K. Weick.  Organizational redesign as improvisation , 1993 .

[32]  C. Ciborra,et al.  Formative contexts and information technology: Understanding the dynamics of innovation in organizations , 1994 .

[33]  Richard T. Vidgen,et al.  A framework for understanding how a unique and local IS development method emerges in practice , 2006, Eur. J. Inf. Syst..

[34]  Andrew Pickering,et al.  The mangle of practice : time, agency, and science , 1997 .

[35]  Peter J. Frost,et al.  Variations on a Theme : Practice Improvisation , 1998 .

[36]  Claudio U. Ciborra,et al.  The labyrinths of Information , 2002 .

[37]  David Lorge Parnas,et al.  A Rational Design Process: How and Why to Fake It , 1985, TAPSOFT, Vol.2.

[38]  A. V. D. Ven,et al.  Using Paradox to Build Management and Organization Theories , 1989 .

[39]  Ilkka Tuomi,et al.  Networks of Innovation , 2002 .

[40]  E. Hutchins.  Cognition in the wild , 1995 .

[41]  W. Orlikowski.  Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations , 2000 .

[42]  M. Cunha,et al.  Organizational improvisation: what, when, how, and why , 1999 .

[43]  T. L. Porte.  High Reliability Organizations: Unlikely, Demanding and At Risk , 1996 .

[44]  Claudio U. Ciborra.  Ciborra “ In the Mood for Knowledge : a New Study of Improvisation , 2001 .

[45]  S. Traweek,et al.  Beamtimes and Lifetimes: The World of High Energy Physicists , 1988 .

[46]  Tony Cornford,et al.  A review of the methodologies movement , 1993, J. Inf. Technol..

[47]  P. Clarke.  A Grid for Particle Physics – from testbed to production , 2004 .

[48]  Hakan Erdogmus,et al.  Scaling Agile Methods , 2003, IEEE Softw..

[49]  Edward Yourdon,et al.  Modern structured analysis , 1989 .

[50]  Kalle Lyytinen,et al.  Systemic Risk, Information Technology Artifacts, and High Reliability Organizations: A Case of Constructing a Radical Architecture , 2004, ICIS.

[51]  Tim Berners-Lee,et al.  Weaving The Web: The Original Design And Ultimate Destiny of the World Wide Web , 1999 .

[52]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[53]  J. Brown,et al.  Organizational Learning and Communities-of-Practice: Toward a Unified View of Working, Learning, and Innovation , 1991 .

[54]  Edwin Hutchins,et al.  Organizing Work by Adaptation , 1991 .

[55]  Giorgio Bruno,et al.  Process-translatable Petri nets for the rapid prototyping of process control systems , 1986, IEEE Transactions on Software Engineering.

[56]  M. Wade,et al.  Review: the resource-based view and information systems research: review, extension, and suggestions for future research , 2004 .

[57]  Jan Pries-Heje,et al.  Short cycle time systems development , 2004, Inf. Syst. J..

[58]  William H. Glick,et al.  Organizational Change and Redesign: Ideas and Insights for Improving Performance , 1995 .

[59]  Tom DeMarco,et al.  Structured Analysis and System Specification (Reprint) , 2002, Software Pioneers.

[60]  Michael X Cohen,et al.  A Garbage Can Model of Organizational Choice. , 1972 .

[61]  Ilkka Tuomi,et al.  The Future of Open Source , 2005 .

[62]  K. Eisenhardt,et al.  PRODUCT DEVELOPMENT: PAST RESEARCH, PRESENT FINDINGS, AND FUTURE DIRECTIONS , 1995 .

[63]  Jos van Hillegersberg,et al.  Aligning IT components to achieve agility in globally distributed system development , 2006, CACM.

[64]  M. Hatch.  Exploring the Empty Spaces of Organizing: How Improvisational Jazz Helps Redescribe Organizational Structure , 1999 .

[65]  Richard Baskerville,et al.  Deep structure or emergence theory: contrasting theoretical foundations for information systems development , 1998, Inf. Syst. J..

[66]  Hans H. Kron,et al.  Programming-in-the-Large Versus Programming-in-the-Small , 1975, IEEE Transactions on Software Engineering.

[67]  Nick V. Flor.  Globally distributed software development and pair programming , 2006, CACM.

[68]  Barry W. Boehm,et al.  A spiral model of software development and enhancement , 1986, Computer.

[69]  K. Knorr-Cetina,et al.  Epistemic cultures : how the sciences make knowledge , 1999 .

[70]  Karl E. Weick,et al.  The aesthetic of imperfection in orchestras and organizations , 2003 .

[71]  K. Weick,et al.  Collective mind in organizations: Heedful interrelating on flight decks. , 1993 .

[72]  Vincent Jordan.  Computers in context: The philosophy and practice of systems design : Bo Dahlbom and Lars Mathiassen NCC Blackwell (1993) 306 pp (inc. bibliography and index) £37.50 hardback, £14.99 paperback ISBN 1 55786 405 5 , 1995, Inf. Softw. Technol..

[73]  Michael R. Wade,et al.  The Resource-Based View and Information Systems Research: Review, Extension, and Suggestions for Future Research , 2004, MIS Q..

[74]  Kathleen M. Sutcliffe,et al.  Special Issue: Frontiers of Organization Science, Part 1 of 2: Organizing and the Process of Sensemaking , 2005, Organ. Sci..

[75]  Giovan Francesco Lanzara,et al.  Between transient constructs and persistent structures: designing systems in action , 1999, J. Strateg. Inf. Syst..

[76]  Mary Crossan,et al.  The improvising organization: Where planning meets opportunity , 1996 .

[77]  J. Kenneth Benson,et al.  Organizations: A Dialectical View. , 1977 .

[78]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[79]  Laurie A. Williams,et al.  Agile software development: it's about feedback and change , 2003, Computer.

[80]  Christine Moorman,et al.  The Convergence of Planning and Execution: Improvisation in New Product Development , 1998 .

[81]  David B. Bills,et al.  The Collective Mind , 2013 .

[82]  Bongsug Chae,et al.  The surface of emergence in systems development: agency, institutions, and large-scale information systems , 2005, Eur. J. Inf. Syst..

[83]  Amany R. Elbanna,et al.  The validity of the improvisation argument in the implementation of rigid technology: the case of ERP systems , 2006, J. Inf. Technol..

[84]  K. WeiK..  The Collapse of Sensemaking in Organizations: The Mann Gulch Disaster , 2009, STUDI ORGANIZZATIVI.

[85]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[86]  Marianne W. Lewis.  Exploring Paradox: Toward a More Comprehensive Guide , 2000 .

[87]  Wendy K. Smith,et al.  Managing Strategic Contradictions: A Top Management Model for Managing Innovation Streams , 2005 .

[88]  K. Weick,et al.  Introductory Essay : Improvisation As a Mindset for Organizational Analysis , 1998 .

[89]  Richard Baskerville.  Artful planning , 2006, Eur. J. Inf. Syst..

[90]  Steve Adolph.  What lessons can the agile community learn from a maverick fighter pilot? , 2006, AGILE 2006 (AGILE'06).

[91]  Bongsug Chae,et al.  The paradoxes of knowledge management: An eastern philosophical perspective , 2006, Inf. Organ..

[92]  C. Ciborra.  The Platform Organization: Recombining Strategies, Structures, and Surprises , 1996 .

[93]  K. Weick.  The Collapse of Sensemaking in Organizations: The Mann Gulch Disaster , 1993 .

[94]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[95]  Varun Grover,et al.  Shaping Agility through Digital Options: Reconceptualizing the Role of Information Technology in Contemporary Firms , 2003, MIS Q..

[96]  David E. Avison,et al.  The fiction of methodological development: a field study of information systems development , 1999, Inf. Technol. People.

[97]  Lars Mathiassen,et al.  Dynamic Capabilities in Small Software Firms: A Sense-and-Respond Approach , 2007, IEEE Transactions on Engineering Management.

[98]  Jim Highsmith,et al.  Agile Software Development Ecosystems , 2002 .

[99]  Tony Cornford,et al.  Introducing pegasus: an ethnographic research project studying the use of grid technologies by the UK particle physics community , 2006 .

[100]  E. Coiera,et al.  Research Paper: Building a National Health IT System from the Middle Out , 2009, J. Am. Medical Informatics Assoc..

[101]  Jan Pries-Heje,et al.  High-Speed Software Development Practices: What Works, What Doesn't , 2006, IT Professional.

[102]  Brian Fitzgerald,et al.  Toward a conceptual framework of agile methods: a study of agility in different disciplines , 2004, WISER '04.

[103]  Lars Mathiassen,et al.  Using Computers in Qualitative Research , 1991 .

[104]  Berend Wierenga,et al.  Improvisation in Action , 1998 .

[105]  Richard Baskerville,et al.  Systems Without Method: The Impact of New Technologies on Information Systems Development Projects , 1992, The Impact of Computer Supported Technologies in Information Systems Development.