Experimental Evaluation of Collaborative Design Features for Process Modelling

Dr. Jan Recker, for their backup in various aspects. First of all, for raising my interest in business process management and giving me the possibility to write my master thesis in-for me-such an interesting topic. Second, for supporting and enabling my research stay abroad at the Queensland University of Technology in Brisbane, Australia. Third, offering me the freedom to explore this topic in my way. This also includes the support given in times when I began to doubt the chosen approach. Your encouragement and trust in me meant very much to me. And not to forget, all the valuable input, ideas and feedback as well as the time for meetings and calls you invested. I know that this is not the normal case and I do not want to take this for granted! Second, I want to thank all people that supported me on my way up until now and made this work possible. In particular, my friends and colleagues for feedback, proofreading and their support. Additionally, thanks to all who participated in the experiments or related pre-studies. Of course, I want to thank my family for being as appreciative and supporting as they were in the last couple of months. At last, I want to thank Julia for her support and appreciation although the time has not been easy for us lately.ABSTRACT Abstract Process modeling is an important design practice in intra-but also in inter-organizational process improvement projects. Inter-organizational process modeling often requires collaboration support for distributed participants. We present the results of seven quasi-experiments of process modeling on basis of collaborative technology. Particularly, we examine a group of process modelers that rely on a collaborative modeling editor to complete a process modeling task in a simulated remote setting. Thereby, we can observe the key phases and tasks of collaborative process modeling, the breakdowns encountered and workarounds employed by the participants. Furthermore, we gather data subject to participants perceptions in order to identify factors influencing the continued usage of collaboration technology support. With our study, we provide a first understanding of the IT-enabled process of process modeling, and detail a set of guidelines and implications for the research and design of collaborative process modeling.

[1]  Mariëlle den Hengst,et al.  Collaborative Modeling of Processes: What Facilitation Support Does a Group Need? , 2005, AMCIS.

[2]  Marcelo Milrad,et al.  Model facilitated learning , 2003 .

[3]  Iris Vessey,et al.  Requirements specification: learning object, process, and data methodologies , 1994, CACM.

[4]  Mariëlle den Hengst,et al.  Collaborative Business Engineering: A Decade of Lessons from the Field , 2004, J. Manag. Inf. Syst..

[5]  Kjeld Schmidt,et al.  Taking CSCW seriously , 1992, Computer Supported Cooperative Work (CSCW).

[6]  Liz Lee-Kelley,et al.  Locus of control and attitudes to working in virtual teams , 2006 .

[7]  Peter Rittgen,et al.  Negotiating Models , 2007, CAiSE.

[8]  Michael Rosemann,et al.  Process Management: A Guide for the Design of Business Processes , 2011 .

[9]  Stijn Hoppenbrouwers,et al.  Fundamental Understanding of the Act of Modelling , 2004 .

[10]  J. Rees TWO CULTURES , 2007, Science.

[11]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[12]  Andrew Clement,et al.  Cooperative support for computer work: a social perspective on the empowering of end users , 1990, CSCW '90.

[13]  Martin J. Eppler,et al.  The Benefits of Synchronous Collaborative Information Visualization: Evidence from an Experimental Evaluation , 2009, IEEE Transactions on Visualization and Computer Graphics.

[14]  E. Tansley,et al.  Using ontology to validate conceptual models , 2003, CACM.

[15]  JoAnne Yates,et al.  Helping CSCW applications succeed: the role of mediators in the context of use , 1994, CSCW '94.

[16]  R. Klassen,et al.  Experimental comparison of Web, electronic and mail survey technologies in operations management , 2001 .

[17]  Mohamed Zairi,et al.  Business process management: a boundaryless approach to modern competitiveness , 1997, Bus. Process. Manag. J..

[18]  Jay F. Nunamaker,et al.  Electronic meeting systems , 1991, CACM.

[19]  Jan Recker,et al.  Continued use of process modeling grammars: the impact of individual difference factors , 2010, Eur. J. Inf. Syst..

[20]  Geert Poels,et al.  Measuring the Perceived Semantic Quality of Information Models , 2005, ER.

[21]  Thomas Kvan,et al.  Collaborative design: what is it? , 2000 .

[22]  Jan Recker,et al.  Context-aware Process Design Exploring the Extrinsic Drivers for Process Flexibility , 2006, BPMDS.

[23]  Jay F. Nunamaker,et al.  Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab and Field Findings , 1996, J. Manag. Inf. Syst..

[24]  Venkataraman Ramesh,et al.  Understanding Conceptual Schemas: Exploring the Role of Application and IS Domain Knowledge , 2006, Inf. Syst. Res..

[25]  H. Klein,et al.  Information systems research: contemporary approaches and emergent traditions , 1991 .

[26]  T. C. Powell Total Quality Management as Competitive Advantage , 1995 .

[27]  Gerhard Fischer,et al.  Transcending the individual human mind—creating shared understanding through collaborative design , 2000, TCHI.

[28]  John D. Murphy,et al.  Avatars, People, and Virtual Worlds: Foundations for Research in Metaverses , 2009, J. Assoc. Inf. Syst..

[29]  Ron Weber,et al.  Research Commentary: Information Systems and Conceptual Modeling - A Research Agenda , 2002, Inf. Syst. Res..

[30]  Peter Rittgen,et al.  Collaborative Modeling - A Design Science Approach , 2009, 2009 42nd Hawaii International Conference on System Sciences.

[31]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[32]  Peter Rittgen Collaborative Design of Models , 2009 .

[33]  Mario Piattini,et al.  Assessing the understandability of UML statechart diagrams with composite states—A family of empirical studies , 2009, Empirical Software Engineering.

[34]  Jan Mendling,et al.  How the Structuring of Domain Knowledge Helps Casual Process Modelers , 2010, ER.

[35]  Mathias Weske,et al.  Business Process Management: A Survey , 2003, Business Process Management.

[36]  Peter Rittgen Collaborative modeling of business processes: a comparative case study , 2009, SAC '09.

[37]  J. Valacich,et al.  Effects of anonymity and evaluative tone on idea generation in computer-mediated groups , 1990 .

[38]  G. Spreitzer An empirical test of a comprehensive model of intrapersonal empowerment in the workplace , 1995, American journal of community psychology.

[39]  Izak Benbasat,et al.  Quo vadis TAM? , 2007, J. Assoc. Inf. Syst..

[40]  B Bader The power of empowerment. , 1999, Michigan health & hospitals.

[41]  Maryam Alavi,et al.  Computer-Mediated Collaborative Learning: An Empirical Evaluation , 1994, MIS Q..

[42]  Peter Rittgen Supporting Electronic Collaboration in Conceptual Modeling , 2008 .

[43]  Jan C. Recker,et al.  Improving the Traversal of Large Hierarchical Process Repositories , 2009 .

[44]  Didar Zowghi,et al.  Requirements Engineering challenges in multi-site software development organizations , 2004 .

[45]  Stijn Hoppenbrouwers,et al.  A Fundamental View on the Process of Conceptual Modeling , 2005, ER.

[46]  Andrew Gemino,et al.  Evaluating modeling techniques based on models of learning , 2003, CACM.

[47]  C. Morris,et al.  Writings on the General Theory of Signs , 1971 .

[48]  Anol Bhattacherjee,et al.  Understanding Information Systems Continuance: An Expectation-Confirmation Model , 2001, MIS Q..

[49]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[50]  Mark A. Fuller,et al.  Facilitator Influence in Group Support Systems: Intended and Unintended Effects , 1998, Inf. Syst. Res..

[51]  Douglas R. Vogel,et al.  Facilitation Methods for Collaborative Modeling Tools , 2000 .

[52]  Marta Indulska,et al.  Business Process Modeling- A Comparative Analysis , 2009, J. Assoc. Inf. Syst..

[53]  Peter Rittgen Collaborative Modeling : A Design Science Approach , 2009 .

[54]  Barbara Gray,et al.  Recognition and reconciliation of differences in interpretation of misalignments when collaborative technologies are introduced into new product development teams , 2003 .

[55]  K. Jöreskog,et al.  LISREL 8: New Statistical Features , 1999 .

[56]  Michael Rosemann,et al.  Application of a Holistic Model for Determining BPM Maturity , 2004 .

[57]  Andrew Gemino,et al.  Complexity and clarity in conceptual modeling: Comparison of mandatory and optional properties , 2005, Data Knowl. Eng..

[58]  Ernesto Arias,et al.  Designing in a design community: insights and challenges , 1995, Symposium on Designing Interactive Systems.

[59]  A. G. Clear International collaborative learning - the facilitation process , 1999 .

[60]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[61]  Mo Adam Mahmood,et al.  Variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature , 2000, Int. J. Hum. Comput. Stud..

[62]  Jay F. Nunamaker,et al.  ELECTRONIC BRAINSTORMING AND GROUP SIZE , 1992 .

[63]  Olivera Marjanovic,et al.  Business Value Creation through Business Processes Management and Operational Business Intelligence Integration , 2010, 2010 43rd Hawaii International Conference on System Sciences.

[64]  J. Recker,et al.  Does It Matter Which Process Modelling Language We Teach or Use? An Experimental Study on Understanding Process Modelling Languages without Formal Education , 2007 .

[65]  Andrew Clement,et al.  Computing at work: empowering action by “low-level users” , 1994, CACM.

[66]  Keng Siau,et al.  Evaluation techniques for systems analysis and design modelling methods – a review and comparative analysis , 2011, Inf. Syst. J..

[67]  Steve Smithson,et al.  An exploratory study of information systems in support of employee empowerment , 2000, J. Inf. Technol..

[68]  Geert Poels,et al.  Evaluating quality of conceptual modelling scripts based on user perceptions , 2007, Data Knowl. Eng..

[69]  R. Maull,et al.  Business Process Re‐engineering , 1994 .

[70]  Wouter R. van Joolingen,et al.  Students' reasoning during modeling in an inquiry learning environment , 2005, Comput. Hum. Behav..

[71]  Howard B. Lee,et al.  A First Course in Factor Analysis 2nd Ed , 1973 .

[72]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[73]  Mark A. Fuller,et al.  Facilitation, systems, and users: the complete socio-technical system , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[74]  Jay F. Nunamaker,et al.  Collaboration Engineering with ThinkLets to Pursue Sustained Success with Group Support Systems , 2003, J. Manag. Inf. Syst..

[75]  Kurt Sandkuhl,et al.  Participative Enterprise Modeling: Experiences and Recommendations , 2007, CAiSE.

[76]  S. Kiesler,et al.  Group decision making and communication technology , 1992 .

[77]  Arne Sølvberg,et al.  Understanding quality in conceptual modeling , 1994, IEEE Software.

[78]  Peter Rittgen,et al.  Success Factors of e-Collaboration in Business Process Modeling , 2010, CAiSE.

[79]  Anol Bhattacherjee,et al.  Explaining information technology usage: A test of competing models☆ , 2008 .

[80]  August-Wilhelm Scheer,et al.  ARIS - Business Process Modeling , 1998 .

[81]  Gerry Stahl,et al.  Computer Support for Collaborative Learning : foundations for a CSCL community : proceedings of CSCL 2002, Boulder, Colorado, USA, January 7-11, 2002 , 2002 .

[82]  Glenn Stewart,et al.  Using Qualitative Analysis for Deriving Evidence Based Construct Definition: A Case Narrative of 'User Empowerment' , 2006 .

[83]  Glenn Stewart,et al.  Exploring the Relationship between User Empowerment and Enterprise System Success Measures , 2004, AMCIS.

[84]  Sylvia A. Ward,et al.  CSCW: The development of a taxonomy , 2004 .

[85]  Paul Harmon EVALUATING AN ORGANIZATIONS BUSINESS PROCESS MATURITY , 2004 .

[86]  Douglas C. Engelbart,et al.  Toward augmenting the human intellect and boosting our collective IQ , 1995, CACM.

[87]  T. Jong,et al.  Supporting simulation-based learning; the effects of model progression and assignments on definitional and intuitive knowledge , 1998 .

[88]  Paul Wilson,et al.  Computer Supported Cooperative Work:: An Introduction , 2014 .

[89]  Jan Mendling,et al.  Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for Correctness , 2008, Lecture Notes in Business Information Processing.

[90]  Morten Kyng,et al.  Design at Work , 1992 .

[91]  Marta Indulska,et al.  Business Process Modeling: Perceived Benefits , 2009, ER.

[92]  Olivera Marjanovic,et al.  Learning and teaching in a synchronous collaborative environment , 1999, J. Comput. Assist. Learn..

[93]  Dov Te'eni,et al.  Modeling as constrained problem solving: an empirical study of the data modeling process , 1995 .

[94]  Björn Niehaves,et al.  Collaborative Business Process Management: Exploring Themes, Achievements, and Perspectives , 2010, ECIS.

[95]  John Jeston,et al.  Business Process Management: Practical Guidelines to Successful Implementations , 2006 .

[96]  Steve Benford,et al.  Collaborative virtual environments , 2001, CACM.

[97]  Andrew Dillon,et al.  Design rationale: Concepts, techniques, and use , 1997 .

[98]  Jonathan Grudin,et al.  Computer-supported cooperative work: history and focus , 1994, Computer.

[99]  Daniela E. Damian,et al.  The impact of stakeholders' geographical distribution on managing requirements in a multi-site organization , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[100]  Jan Mendling,et al.  Seven process modeling guidelines (7PMG) , 2010, Inf. Softw. Technol..

[101]  R. Stamper The semiotic framework for informations systems research , 1991 .

[102]  Peter Meso,et al.  The Effects of Decomposition Quality and Multiple Forms of Information on Novices' Understanding of a Domain from a Conceptual Model , 2008, J. Assoc. Inf. Syst..

[103]  Wil M.P. van der Aalst,et al.  Process mining: discovering workflow models from event-based data , 2001 .

[104]  Andrew Gemino,et al.  Using Iconic Graphics in Entity-Relationship Diagrams: The Impact on Understanding , 2008, J. Database Manag..

[105]  Jano Moreira de Souza,et al.  Assisting meeting facilitation through automated analysis of group dynamics , 2008, 2008 12th International Conference on Computer Supported Cooperative Work in Design.

[106]  Theo P. van der Weide,et al.  Information modeling: The process and the required competencies of its participants , 2004, Data Knowl. Eng..

[107]  James D. Palmer,et al.  A multimedia computer supported cooperative work environment for requirements engineering , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[108]  Colin Armistead,et al.  Business process management - lessons from European business , 1999, Bus. Process. Manag. J..

[109]  Jan Recker,et al.  Opportunities and constraints: the current struggle with BPMN , 2010, Bus. Process. Manag. J..

[110]  Peter Rittgen 19P. Collaborative Business Process Modeling – Tool Support for Solving Typical Problems , 2010 .

[111]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[112]  Jonathan Grudin,et al.  Supporting Indirect Collaborative Design With Integrated Knowledge-Based Design Environments , 1992, Hum. Comput. Interact..

[113]  Andrew Gemino,et al.  A framework for empirical evaluation of conceptual modeling techniques , 2004, Requirements Engineering.

[114]  Bill Curtis,et al.  Breakdowns and processes during the early activities of software design by professionals , 1987 .

[115]  Lori A. Pacini The Power of Empowerment. , 2000 .

[116]  P. Harmon Business Process Change: A Guide for Business Managers and BPM and Six Sigma Professionals , 2007 .

[117]  Stijn Hoppenbrouwers,et al.  Interactions, Goals and Rules in a Collaborative Modelling Session , 2009, PoEM.

[118]  Weiming Shen,et al.  Collaborative conceptual design - state of the art and future trends , 2002, Comput. Aided Des..

[119]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[120]  Ronald M. Baecker,et al.  Readings in human-computer interaction : toward the year 2000 , 1995 .

[121]  R. Bostrom,et al.  Group Facilitation and Group Support Systems , 1993 .

[122]  Hongyan Ma,et al.  Process-aware information systems: Bridging people and software through process technology , 2007, J. Assoc. Inf. Sci. Technol..

[123]  B. Wilpert,et al.  Conceptual dimensions and boundaries of participation in organizations: a critical evaluation. , 1978, Administrative science quarterly.

[124]  B. Price A First Course in Factor Analysis , 1993 .

[125]  Guy G. Gable,et al.  Integrating case study and survey research methods: an example in information systems , 1994 .

[126]  P. Mattessich,et al.  Collaboration: What Makes It Work- A Review of Research Literature on Factors Influencing Successful Collaboration , 1992 .

[127]  Theo P. van der Weide,et al.  Information modeling: The process and the required competencies of its participants , 2006, Data Knowl. Eng..

[128]  Audris Mockus,et al.  Formulation and preliminary test of an empirical theory of coordination in software engineering , 2003, ESEC/FSE-11.

[129]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[130]  J. Becker,et al.  The Process in Focus , 2003 .

[131]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[132]  J. Frederiksen,et al.  A Theoretical Framework and Approach for Fostering Metacognitive Development , 2005 .

[133]  James Boyle,et al.  Factors influencing the success of computer mediated communication (CMC) environments in university teaching: a review and case study , 2000, Comput. Educ..

[134]  Mathias Weske,et al.  Business Process Management: Concepts, Languages, Architectures , 2007 .

[135]  Don A. Dillman,et al.  Development of a Standard E-Mail Methodology: Results of an Experiment , 1998 .

[136]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[137]  Gloria Mark,et al.  Conventions and Commitments in Distributed CSCW Groups , 2002, Computer Supported Cooperative Work (CSCW).

[138]  Jan Mendling,et al.  A Discourse on Complexity of Process Models , 2006, Business Process Management Workshops.

[139]  Stijn Hoppenbrouwers,et al.  Exploring Modelling Strategies in a Meta-modelling Context , 2006, OTM Workshops.

[140]  Peter B. Seddon A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[141]  Hajo A. Reijers,et al.  Social software for business process modeling , 2010, J. Inf. Technol..

[142]  Andrew Blyth,et al.  Business process re-engineering: What is it? , 1997, SIGG.

[143]  Robert Johansen,et al.  Groupware: Computer Support for Business Teams , 1988 .

[144]  Peter Rittgen Quality and perceived usefulness of process models , 2010, SAC '10.

[145]  Peter Gyngell,et al.  Process Innovation: Reengineering Work through Information Technology , 1994 .

[146]  Adam Mahmoodmo,et al.  Variables affecting information technology end-user satisfaction , 2000 .