The Last Research Mile: Achieving Both Rigor and Relevance in Information Systems Research

Abstract From our desk chairs it may be tempting to work up an idea, build a quick prototype, test it in a lab, and say, “Our work here is done; the rest is merely details.” More scholarly knowledge awaits discovery, however, by researchers who shepherd an information systems (IS) solution through the last research mile, that is, through successful transition to the workplace. Going the last research mile means using scientific knowledge and methods to address important unsolved classes of problems for real people with real stakes in the outcomes. The last research mile proceeds in three stages: proof-of-concept research to demonstrate the functional feasibility of a solution; proof-of-value research to investigate whether a solution can create value across a variety of conditions; and proof-of-use research to address complex issues of operational feasibility. The last research mile ends only when practitioners routinely use a solution in the field. We argue that going the last research mile negates the assumption that one must trade off rigor and relevance, showing it to be it a false dilemma. Systems researchers who take their solutions through the last research mile may ultimately have the greatest impact on science and society. We demonstrate the last research mile with cases from our own work and the work of others spanning more than forty years.

[1]  David Kirkpatrick,et al.  Here comes the payoff from PCs , 1993 .

[2]  Jay F. Nunamaker,et al.  ELECTRONIC BRAINSTORMING AND GROUP SIZE , 1992 .

[3]  Jay F. Nunamaker,et al.  Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab and Field Findings , 1996, J. Manag. Inf. Syst..

[4]  R. Stebbins Exploratory research in the social sciences , 2001 .

[5]  Bjørn Erik Munkvold,et al.  Organizational adoption and diffusion of electronic meeting systems: a case study , 2001, GROUP.

[6]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[7]  Brad Quinn Post,et al.  A Business Case Framework for Group Support Technology , 1992, J. Manag. Inf. Syst..

[8]  Jay F. Nunamaker,et al.  Electronic meeting systems , 1991, CACM.

[9]  Jay F. Nunamaker,et al.  Establishing a foundation for automated human credibility screening , 2012, 2012 IEEE International Conference on Intelligence and Security Informatics.

[10]  Robert O. Briggs,et al.  Evaluating the Performance of Collaboration Engineers , 2015, 2015 48th Hawaii International Conference on System Sciences.

[11]  Robert O. Briggs,et al.  Towards an Instrument to Measure Successfulness of Collaborative Effort from a Participant Perspective , 2009, 2009 42nd Hawaii International Conference on System Sciences.

[12]  John D. Murphy,et al.  Discovering and Evaluating Collaboration Engineering Opportunities: An Interview Protocol Based on the Value Frequency Model , 2011 .

[13]  Matthias Gamer,et al.  Memory Detection: Detecting concealed information using autonomic measures , 2011 .

[14]  Robert O. Briggs,et al.  Facilitator-in-a-Box: Process Support Applications to Help Practitioners Realize the Potential of Collaboration Technology , 2013, J. Manag. Inf. Syst..

[15]  Jay F. Nunamaker,et al.  Facilitating Group Creativity: Experience with a Group Decision Support System , 1987, J. Manag. Inf. Syst..

[16]  J. Burgoon,et al.  Deceptive Realities , 1996 .

[17]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[18]  J. Valacich,et al.  Effects of anonymity and evaluative tone on idea generation in computer-mediated groups , 1990 .

[19]  John J. Sosik,et al.  Effects of leadership style and anonymity on group potency and effectiveness in a group decision support system environment. , 1997 .

[20]  Robert O. Briggs,et al.  A conceptual foundation of the thinkLet concept for Collaboration Engineering , 2006, Int. J. Hum. Comput. Stud..

[21]  B. E. Wynne,et al.  An experiment assessing group support system and facilitator effects on meeting outcomes , 1995 .

[22]  R Yin,et al.  Case study research designs and methods , 2014 .

[23]  Jay F. Nunamaker,et al.  On the Measurement of Ideation Quality , 2007, J. Manag. Inf. Syst..

[24]  Robert O. Briggs,et al.  Social loafing in electronic brainstorming: invoking social comparison through technology and facilitation techniques to improve group productivity , 1995, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences.

[25]  Jay F. Nunamaker,et al.  Computer-Aided Deliberation: Model Management and Group Decision Support: Special Focus Article , 1988, Oper. Res..

[26]  Starr Roxanne Hiltz,et al.  Group Support Systems: A Descriptive Evaluation of Case and Field Studies , 2000, J. Manag. Inf. Syst..

[27]  Robert O. Briggs,et al.  Collaboration Engineering: Foundations and Opportunities: Editorial to the Special Issue on the Journal of the Association of Information Systems , 2009, J. Assoc. Inf. Syst..

[28]  Inu Sarah,et al.  “ Enforced ” vs . “ casual ” transparency-Findings from IT-supported financial advisory encounters , 2017 .

[29]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[30]  Robert O. Briggs,et al.  Surfacing tacit knowledge in requirements negotiation: experiences using EasyWinWin , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[31]  Robert O. Briggs,et al.  A Six-Layer Model of Collaboration , 2015 .

[32]  Ernest A. Hershey,et al.  PSL/PSA: A Computer-Aided Technique for Structured Documentation and Analysis of Information Processing Systems , 1976, IEEE Transactions on Software Engineering.

[33]  Alan R. Dennis,et al.  Process Structuring in Electronic Brainstorming , 1996, Inf. Syst. Res..

[34]  Jay F. Nunamaker,et al.  Collaboration Engineering with ThinkLets to Pursue Sustained Success with Group Support Systems , 2003, J. Manag. Inf. Syst..

[35]  Robert O. Briggs,et al.  AIS Electronic Library (AISeL) , 2018 .

[36]  Robert O. Briggs,et al.  A Tale of Two Cities: Case Studies of Group Support Systems Transition , 2005 .

[37]  Robert O. Briggs,et al.  The Seven Layer Model of Collaboration: An Exploratory Study of Process Identification and Improvement , 2012, 2012 45th Hawaii International Conference on System Sciences.

[38]  Gerardine DeSanctis,et al.  Computer-Based Support for Group Problem-Finding: An Experimental Investigation , 1988, MIS Q..

[39]  Robert O. Briggs,et al.  The Yield Shift Theory of Satisfaction and Its Application to the IS/IT Domain , 2008, J. Assoc. Inf. Syst..

[40]  P. Barrouillet,et al.  The Time-Based Resource-Sharing Model of Working Memory , 2020, Working Memory.

[41]  Alan R. Hevner,et al.  Design Science Research: Looking to the Future , 2010 .

[42]  Jay F. Nunamaker,et al.  Toward a broader vision for Information Systems , 2011, TMIS.

[43]  Jay F. Nunamaker,et al.  Collaborative Cross-Border Security Infrastructure and Systems: Identifying Policy, Managerial and Technological Challenges , 2013, Int. J. E Politics.

[44]  Robert O. Briggs,et al.  Bounded Ideation Theory , 2010, J. Manag. Inf. Syst..

[45]  Jeffrey T. Hancock,et al.  On Lying and Being Lied To: A Linguistic Analysis of Deception in Computer-Mediated Communication , 2007 .

[46]  Simon S. K. Lam,et al.  The Effects of Group Decision Support Systems and Task Structures on Group Communication and Decision Quality , 1997, J. Manag. Inf. Syst..

[47]  Anol Bhattacherjee,et al.  Understanding Information Systems Continuance: An Expectation-Confirmation Model , 2001, MIS Q..

[48]  Jay F. Nunamaker,et al.  A Comparison of Laboratory and Field Research in the Study of Electronic Meeting Systems , 1990, J. Manag. Inf. Syst..

[49]  M. Diehl,et al.  Productivity loss in brainstorming groups: Toward the solution of a riddle. , 1987 .

[50]  A. Osborn Applied Imagination: Principles and Procedures of Creative Thinking , 1953 .

[51]  Jay F. Nunamaker,et al.  Implementing Electronic Meeting Systems at IBM: Lessons Learned and Success Factors , 1990, MIS Q..

[52]  Gerhard Schwabe,et al.  “Enforced” vs. “Casual” Transparency -- Findings from IT-Supported Financial Advisory Encounters , 2012, TMIS.

[53]  A. Noel Jones,et al.  Using technology for stakeholder consultation. The World Bank's use of GroupSystems V in developing countries , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[54]  Samuel B. Bacharach,et al.  Organizational Theories: Some Criteria for Evaluation , 1989 .

[55]  Robert O. Briggs,et al.  A Seven-Layer Model of Collaboration: Separation of Concerns for Designers of Collaboration Systems , 2009, ICIS.

[56]  Jay F. Nunamaker,et al.  ThinkLets: achieving predictable, repeatable patterns of group interaction with group support systems (GSS) , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[57]  Paul Grünbacher Collaborative requirements negotiation with EasyWinWin , 2000, Proceedings 11th International Workshop on Database and Expert Systems Applications.

[58]  Jay F. Nunamaker,et al.  Computer-aided analysis and design of information systems , 1976, CACM.

[59]  Robert O. Briggs,et al.  A tale of two cities: case studies of GSS transition in two organizations , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[60]  Robert O. Briggs,et al.  Causal Relationships in Creative Problem Solving: Comparing Facilitation Interventions for Ideation , 2004, J. Manag. Inf. Syst..

[61]  Gerhard Schwabe,et al.  Enabling relationship building in tabletop-supported advisory settings , 2014, CSCW.

[62]  Barry W. Boehm,et al.  EasyWinWin: a groupware-supported methodology for requirements negotiation , 2001, ESEC/FSE-9.

[63]  Thomas Lee Rodgers,et al.  Identifying Quality, Novel, and Creative Ideas: Constructs and Scales for Idea Evaluation , 2006, J. Assoc. Inf. Syst..

[64]  R. Yin Yin, Robert K., Case Study Research: Design and Methods, 2nd ed. Newbury Park, CA: Sage, 1994. , 1994 .

[65]  Jay F. Nunamaker,et al.  PLEXSYS-84: An Integrated Development Environment for Informational Systems , 1984, J. Manag. Inf. Syst..

[66]  Jay F. Nunamaker,et al.  Border Security Credibility Assessments via Heterogeneous Sensor Fusion , 2010, IEEE Intelligent Systems.

[67]  Steven L. Alter Work System Theory: Overview of Core Concepts, Extensions, and Challenges for the Future , 2013, J. Assoc. Inf. Syst..

[68]  J. Perry,et al.  Conceptualizing and Measuring Collaboration , 2007 .

[69]  Robert O. Briggs,et al.  Coding for Unique Ideas and Ambiguity: A Method for Measuring the Effect of Convergence on the Artifact of an Ideation Activity , 2011, Int. J. Soc. Organ. Dyn. IT.

[70]  Robert O. Briggs,et al.  On Expanding the Scope of Design Science in IS Research , 2011, DESRIST.

[71]  Jay F. Nunamaker,et al.  Experience at IBM with group support systems: A field study , 1989, Decis. Support Syst..

[72]  Jay F. Nunamaker,et al.  Design Principles for Special Purpose, Embodied, Conversational Intelligence with Environmental Sensors (SPECIES) Agents , 2011 .