Ieee Transactions on Software Engineering, Manuscript Id Stakerare: Using Social Networks and Collaborative Filtering for Large-scale Requirements Elicitation

Requirements elicitation is the software engineering activity in which stakeholder needs are understood. It involves identifying and prioritizing requirements-a process difficult to scale to large software projects with many stakeholders. This paper proposes StakeRare, a novel method that uses social networks and collaborative filtering to identify and prioritize requirements in large software projects. StakeRare identifies stakeholders and asks them to recommend other stakeholders and stakeholder roles, builds a social network with stakeholders as nodes and their recommendations as links, and prioritizes stakeholders using a variety of social network measures to determine their project influence. It then asks the stakeholders to rate an initial list of requirements, recommends other relevant requirements to them using collaborative filtering, and prioritizes their requirements using their ratings weighted by their project influence. StakeRare was evaluated by applying it to a software project for a 30,000-user system, and a substantial empirical study of requirements elicitation was conducted. Using the data collected from surveying and interviewing 87 stakeholders, the study demonstrated that StakeRare predicts stakeholder needs accurately and arrives at a more complete and accurately prioritized list of requirements compared to the existing method used in the project, taking only a fraction of the time.

[1]  Joachim Karlsson,et al.  Software requirements prioritizing , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[2]  Betty H. C. Cheng,et al.  Research Directions in Requirements Engineering , 2007, Future of Software Engineering (FOSE '07).

[3]  Donald D. Tippett,et al.  Project Risk Management Using the Project Risk FMEA , 2004 .

[4]  Colin Hood,et al.  Requirements Management: Interface Between Requirements Development and all other Engineering Processes , 2007 .

[5]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994, Structural analysis in the social sciences.

[6]  F. Apel,et al.  Nice to Have , 1998 .

[7]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[8]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[9]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[10]  Jim Arlow,et al.  Enterprise Patterns and MDA: Building Better Software with Archetype Patterns and UML , 2003 .

[11]  A. Herrmann,et al.  Requirements Prioritization Based on Benefit and Cost Prediction: An Agenda for Future Research , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[12]  Toby Segaran,et al.  Programming Collective Intelligence , 2007 .

[13]  Trevor Wood-Harper,et al.  Understanding the Sources of Information Systems Project Failure , 2007 .

[14]  Jerome L. Myers,et al.  Research Design and Statistical Analysis , 1991 .

[15]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[16]  Joseph A. Goguen,et al.  Requirements engineering: social and technical issues , 1994 .

[17]  Audris Mockus,et al.  Expertise Browser: a quantitative approach to identifying expertise , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[18]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[19]  Martin Höst,et al.  An Industrial Case Study on Distributed Prioritisation in Market-Driven Requirements Engineering for Packaged Software , 2001, Requirements Engineering.

[20]  Ian Sommerville,et al.  Lessons from the Failure and Subsequent Success of a Complex Healthcare Sector IT Project , 2010, ArXiv.

[21]  Neil A. M. Maiden,et al.  Provoking Creativity: Imagine What Your Requirements Could Be Like , 2004, IEEE Softw..

[22]  Roel Wieringa,et al.  Guest Editors' Introduction: Stakeholders in Requirements Engineering , 2007, IEEE Software.

[23]  Julie Greensmith,et al.  Immune system approaches to intrusion detection – a review , 2004, Natural Computing.

[24]  Pamela Zave Classification of research efforts in requirements engineering , 1997, ACM Comput. Surv..

[25]  Nicolas Marmaras,et al.  Moving Target: Designing for Evolving Practice , 2003, HCI.

[26]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[27]  Dean Leffingwell,et al.  Managing Software Requirements: A Use Case Approach , 2003 .

[28]  Edmund A. Mennis The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations , 2006 .

[29]  Stanley N. Salthe,et al.  Development and Evolution: Complexity and Change in Biology , 1993 .

[30]  T. Capers Jones,et al.  Estimating software costs , 1998 .

[31]  Andrew M. Kuhn,et al.  Code Complete , 2005, Technometrics.

[32]  Mikio Aoyama,et al.  Persona-and-scenario based requirements engineering for software embedded in digital consumer products , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[33]  B. Flyvbjerg Five Misunderstandings About Case-Study Research , 2006, 1304.1186.

[34]  Vladan Devedzic,et al.  Software Project Management , 2001 .

[35]  Tom Mens,et al.  4th Workshop on Object-Oriented Architectural Evolution , 2001, ECOOP Workshops.

[36]  Donald MacKenzie,et al.  Mechanizing Proof: Computing, Risk, and Trust , 2001 .

[37]  Ian Sommerville,et al.  Requirements Engineering: Processes and Techniques , 1998 .

[38]  Didar Zowghi,et al.  Requirements Engineering challenges in multi-site software development organizations , 2004 .

[39]  Julio Cesar Sampaio do Prado Leite,et al.  On Non-Functional Requirements in Software Engineering , 2009, Conceptual Modeling: Foundations and Applications.

[40]  Joseph W. Yoder,et al.  Big Ball of Mud , 1997 .

[41]  Kristin A. Farry,et al.  Customer Centered Products: Creating Successful Products Through Smart Requirements Management (Продукты для клиента: создание успешных продуктов с помощью грамотного управления требованиями) , 2000 .

[42]  R. F. Heckard,et al.  Statistical Ideas And Methods , 2005 .

[43]  Natalia Juristo Juzgado,et al.  Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[44]  Anita Sarma,et al.  Tesseract: Interactive visual exploration of socio-technical relationships in software development , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[45]  Jörg Rech,et al.  Wiki-Based Stakeholder Participation in Requirements Engineering , 2007, IEEE Software.

[46]  Ian Simmonds,et al.  A Shearing Layers Approach to Information Systems Development , 2000 .

[47]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[48]  Moshe Ben-Bassat,et al.  A user's approach to requirements analysis of a large software system , 1984, ACM '84.

[49]  Jung-Won Park Supporting Distributed Collaborative Prioritization for WinWin Requirements Capture and Negotiations , 1999 .

[50]  Jane Cleland-Huang,et al.  Towards Automated Requirements Triage , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[51]  Max Chevalier,et al.  Collaborative and Social Information Retrieval and Access: Techniques for Improved User Modeling , 2008 .

[52]  C. Eden,et al.  Making Strategy: The Journey of Strategic Management , 1998 .

[53]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[54]  Randy K. Smith,et al.  Value-Oriented Requirements Prioritization in a Small Development Organization , 2007, IEEE Software.

[55]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[56]  Neil A. M. Maiden,et al.  ACRE: selecting methods for requirements acquisition , 1996, Softw. Eng. J..

[57]  Brian Berenbach,et al.  Software & Systems Requirements Engineering: In Practice , 2009 .

[58]  Galal H Galal,et al.  Requirements engineering: A good practice , 2000 .

[59]  Axel van Lamsweerde,et al.  Formal refinement patterns for goal-driven requirements elaboration , 1996, SIGSOFT '96.

[60]  Jane Cleland-Huang,et al.  Using data mining and recommender systems to scale up the requirements process , 2008, ULSSIS '08.

[61]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[62]  Colin Bentley,et al.  PRINCE2: A Practical Handbook , 2009 .

[63]  Mark E. J. Newman A measure of betweenness centrality based on random walks , 2005, Soc. Networks.

[64]  Philippe Kruchten,et al.  Architecture blueprints—the “4+1” view model of software architecture , 1995, TRI-Ada '95.

[65]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[66]  R. Freeman Strategic Management: A Stakeholder Approach , 2010 .

[67]  Floyd J. Fowler,et al.  Survey Research Methods , 1984 .

[68]  Alan M. Davis,et al.  Operational prototyping: a new development approach , 1992, IEEE Software.

[69]  Stuart Anderson,et al.  Requirements Evolution from Process to Product Oriented Management , 2001, PROFES.

[70]  Barry W. Boehm,et al.  Guest Editors' Introduction: Software Risk Management , 1997, IEEE Softw..

[71]  Daniela E. Damian,et al.  Guest Editors' Introduction: Global Software Development: How Far Have We Come? , 2006, IEEE Software.

[72]  Denise Silver,et al.  Joint Application Development , 1995 .

[73]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[74]  Eric S. K. Yu,et al.  Towards modelling and reasoning support for early-phase requirements engineering , 1997, Proceedings of ISRE '97: 3rd IEEE International Symposium on Requirements Engineering.

[75]  Martin Maguire,et al.  User Requirements Analysis , 2002 .

[76]  Blaize Horner Reich,et al.  The impact of size and volatility on IT project performance , 2007, CACM.

[77]  Laurie A. Williams,et al.  Predicting failures with developer networks and social network analysis , 2008, SIGSOFT '08/FSE-16.

[78]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[79]  Daniel Galin,et al.  SOFTWARE QUALITY ASSURANCE , 2008 .

[80]  Daniela E. Damian,et al.  StakeSource2.0: using social networks of stakeholders to identify and prioritise requirements , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[81]  John Scott Social Network Analysis , 1988 .

[82]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[83]  John B. Kidd,et al.  Decisions with Multiple Objectives—Preferences and Value Tradeoffs , 1977 .

[84]  Marianne LaFrance,et al.  The Knowledge Acquisition Grid: A Method for Training Knowledge Engineers , 1987, Int. J. Man Mach. Stud..

[85]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[86]  Frank Moisiadis,et al.  The fundamentals of prioritising requirements , 2002 .

[87]  R. O'Neill A Hierarchical Concept of Ecosystems. , 1986 .

[88]  Richard T. Dué,et al.  OBJECT-ORIENTED TECHNOLOGY: The Economics of a New Paradigm , 1993 .

[89]  Ulrik Brandes,et al.  On variants of shortest-path betweenness centrality and their generic computation , 2008, Soc. Networks.

[90]  Paul G. Hoel,et al.  Elementary statistics , 1971 .

[91]  Barry W. Boehm,et al.  Theory-W Software Project Management: Principles and Examples , 1989, IEEE Trans. Software Eng..

[92]  Jane Cleland-Huang,et al.  Enhancing Stakeholder Profiles to Improve Recommendations in Online Requirements Elicitation , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[93]  Robert Schaefer,et al.  A systems analysis of systems integration , 2008, SOEN.

[94]  Yehuda Koren,et al.  Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[95]  Daniel Galin,et al.  Software Quality Assurance: From Theory to Implementation , 2003 .

[96]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[97]  Colin Potts Metaphors of intent , 2001, Proceedings Fifth IEEE International Symposium on Requirements Engineering.

[98]  Chris F. Kemerer,et al.  Reliability of function points measurement: a field experiment , 2015, CACM.

[99]  Jane Cleland-Huang,et al.  Towards automated requirements prioritization and triage , 2009, Requirements Engineering.

[100]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[101]  D. Ross Jeffery,et al.  Function Points in the Estimation and Evaluation of the Software Process , 1990, IEEE Trans. Software Eng..

[102]  Dan Frankowski,et al.  Collaborative Filtering Recommender Systems , 2007, The Adaptive Web.

[103]  Daniele Quercia,et al.  StakeSource: harnessing the power of crowdsourcing and social networks in stakeholder analysis , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[104]  Capers Jones Applied Software Measurement: Global Analysis of Productivity and Quality , 1991 .

[105]  Capers Jones,et al.  Why software fails , 1996 .

[106]  Robert Schaefer,et al.  A rational theory of system-making systems , 2006, SOEN.

[107]  Daniela E. Damian,et al.  Collaboration Patterns and the Impact of Distance on Awareness in Requirements-Centred Social Networks , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[108]  Capers Jones,et al.  Patterns of Large Software Systems: Failure and Success , 1995, Computer.

[109]  Barbara Farbey,et al.  Exploring Requirements: Quality Before Design , 1991 .

[110]  Philippe Krutchen,et al.  Architectural blueprints--the '4+1' view model of software architecture , 1995 .

[111]  Alan M. Davis,et al.  Software Requirements: Objects, Functions and States , 1993 .

[112]  Philippe Kruchten,et al.  The 4+1 View Model of Architecture , 1995, IEEE Softw..

[113]  Barry W. Boehm,et al.  Software Risk Management , 1989, ESEC.

[114]  Mike Kuniavsky,et al.  Observing the User Experience: A Practitioner's Guide to User Research (Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies) , 2003 .

[115]  Walker Royce Software Project Management , 1998 .

[116]  Ken-ichi Matsumoto,et al.  Accelerating cross-project knowledge collaboration using collaborative filtering and social networks , 2005, MSR.

[117]  Vijay Sivaraman,et al.  UNIVERSITY OF NEW SOUTH WALES , 1967 .

[118]  Neal Lathia,et al.  Computing Recommendations with Collaborative Filtering , 2009 .

[119]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[120]  Marco Kuhrmann,et al.  Global Software Engineering , 2010 .

[121]  Jane Cleland-Huang,et al.  A recommender system for requirements elicitation in large-scale software projects , 2009, SAC '09.

[122]  Dean Leffingwell,et al.  Managing software requirements: a unified approach , 1999 .

[123]  Patrik Berander,et al.  Hierarchical Cumulative Voting (hcv) - Prioritization of Requirements in Hierarchies , 2006, Int. J. Softw. Eng. Knowl. Eng..

[124]  M. V. Didkovska,et al.  Software project management , 2011, 2011 21st International Crimean Conference "Microwave & Telecommunication Technology".

[125]  Joachim Karlsson,et al.  A Cost-Value Approach for Prioritizing Requirements , 1997, IEEE Softw..

[126]  Daniela E. Damian,et al.  Predicting build failures using social network analysis on developer communication , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[127]  George E. Stark,et al.  An examination of the effects of requirements changes on software maintenance releases , 1999 .

[128]  Ian F. Alexander,et al.  A Taxonomy of Stakeholders: Human Roles in System Development , 2005, Int. J. Technol. Hum. Interact..

[129]  Ping Zhang Model Selection Via Multifold Cross Validation , 1993 .

[130]  Satnam Alag,et al.  Collective Intelligence in Action , 2008 .

[131]  M. Kendall Elementary Statistics , 1945, Nature.

[132]  Qiang Yang,et al.  Scalable collaborative filtering using cluster-based smoothing , 2005, SIGIR '05.

[133]  R. Hanneman Introduction to Social Network Methods , 2001 .

[134]  Geoff Easton,et al.  Industrial Networks: A New View of Reality , 1992 .

[135]  Michael E. Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[136]  H. Goldstein,et al.  Who killed the virtual case file? [case management software] , 2005, IEEE Spectrum.

[137]  Martin S. Feather,et al.  Requirements and Specification Exemplars , 1997, Automated Software Engineering.

[138]  John Scott What is social network analysis , 2010 .

[139]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[140]  Suzanne Robertson,et al.  Mastering the Requirements Process , 1999 .

[141]  Douglas C. Schmidt,et al.  Ultra-Large-Scale Systems: The Software Challenge of the Future , 2006 .

[142]  Lin Liu,et al.  Designing information systems in social context: a goal and scenario modelling approach , 2004, Inf. Syst..

[143]  D. Ross Jeffery,et al.  An Inter-Organizational Comparison of Programming Productivity , 1979, ICSE.

[144]  Jr. Frederick P. Brooks,et al.  The mythical man-month (anniversary ed.) , 1995 .

[145]  Luo Si,et al.  An automatic weighting scheme for collaborative filtering , 2004, SIGIR '04.

[146]  Colette Rolland,et al.  Guiding Goal Modeling Using Scenarios , 1998, IEEE Trans. Software Eng..

[147]  Axel van Lamsweerde,et al.  Goal-Oriented Requirements Engineering: A Guided Tour , 2001, RE.

[148]  Marjo Kauppinen,et al.  Requirements Prioritization Challenges in Practice , 2004, PROFES.

[149]  Mordechai Ben-Menachem,et al.  Writing effective use cases , 2001, SOEN.

[150]  Mary Shaw,et al.  Prospects for an engineering discipline of software , 1990, IEEE Software.

[151]  Jesús M. González-Barahona,et al.  Applying Social Network Analysis to the Information in CVS Repositories , 2004, MSR.

[152]  S French,et al.  Multicriteria Methodology for Decision Aiding , 1996 .

[153]  A. Finkelstein,et al.  A comedy of errors: the London Ambulance Service case study , 1996, Proceedings of the 8th International Workshop on Software Specification and Design.

[154]  Denise Johnson McManus,et al.  Stakeholder Risk Assessment: An Outcome-Based Approach , 2007, IEEE Software.

[155]  S.D.P. Harker,et al.  The change and evolution of requirements as a challenge to the practice of software engineering , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[156]  Annie I. Antón,et al.  Goal-based requirements analysis , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[157]  Neil A. M. Maiden,et al.  End-user requirements blogging with iRequire , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[158]  Michael G. Madden,et al.  The effect of principal component analysis on machine learning accuracy with high-dimensional spectral data , 2005, Knowl. Based Syst..

[159]  John Grundy,et al.  Collaborative Software Engineering , 2010 .

[160]  A. Pouloudi,et al.  Stakeholder identification in inter-organizational systems: gaining insights for drug use management systems , 1997 .

[161]  H. Cooper,et al.  A Quantitative Review of Research Design Effects on Response Rates to Questionnaires , 1983 .

[162]  Chris Anderson,et al.  The Long Tail: Why the Future of Business is Selling Less of More , 2006 .

[163]  Axel van Lamsweerde,et al.  Agent-based tactics for goal-oriented requirements elaboration , 2002, ICSE '02.

[164]  Daniela E. Damian,et al.  Requirements-Driven Collaboration: Leveraging the Invisible Relationships between Requirements and People , 2010, Collaborative Software Engineering.

[165]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[166]  Yijun Yu,et al.  From goals to aspects: discovering aspects from requirements goal models , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[167]  Neil A. M. Maiden,et al.  CREWS-SAVRE: Scenarios for Acquiring and Validating Requirements , 1998, Automated Software Engineering.

[168]  H. Holbrook,et al.  A scenario-based methodology for conducting requirements elicitation , 1990, SOEN.

[169]  Karl E. Wiegers First Things First: Prioritizing Requirements , 1999 .

[170]  Harald C. Gall,et al.  Journal of Software Maintenance and Evolution: Research and Practice Visualizing Feature Evolution of Large-scale Software Based on Problem and Modification Report Data , 2022 .

[171]  Alfred Kobsa,et al.  The Adaptive Web, Methods and Strategies of Web Personalization , 2007, The Adaptive Web.

[172]  Michael Jackson,et al.  Four dark corners of requirements engineering , 1997, TSEM.

[173]  Priscilla S. Markwood,et al.  The Long Tail: Why the Future of Business is Selling Less of More , 2006 .

[174]  Yoji Akao,et al.  Quality Function Deployment : Integrating Customer Requirements into Product Design , 1990 .

[175]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[176]  Horst Lichter,et al.  Prototyping in industrial software projects—bridging the gap between theory and practice , 1994, ICSE '93.

[177]  Stephen J. Andriole Storyboard prototyping: a new approach to user requirements analysis , 1989 .

[178]  Galal H. Galal-Edeen,et al.  Requirements engineering: A good practice , 2000, Eur. J. Inf. Syst..

[179]  Anne Bruseberg,et al.  New product development by eliciting user experience and aspirations , 2001, Int. J. Hum. Comput. Stud..

[180]  Nachiappan Nagappan,et al.  Predicting defects using network analysis on dependency graphs , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[181]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[182]  Susan P. Williams,et al.  Using card sorting technique to classify requirements change , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[183]  Axel van Lamsweerde,et al.  Requirements Engineering: From System Goals to UML Models to Software Specifications , 2009 .

[184]  Daniele Quercia,et al.  StakeNet: using social networks to analyse the stakeholders of large-scale software projects , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[185]  S. Lauesen Software Requirements Styles and Techniques , 2001 .

[186]  Alan M. Davis,et al.  The Art of Requirements Triage , 2003, Computer.

[187]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[188]  Thomas R. Lindlof Qualitative Communication Research Methods , 1994 .

[189]  Philippe Massonet,et al.  Goal-directed elaboration of requirements for a meeting scheduler: problems and lessons learnt , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[190]  Danah Boyd,et al.  Vizster: visualizing online social networks , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[191]  C. Prell,et al.  Stakeholder Analysis and Social Network Analysis in Natural Resource Management , 2009, Society & Natural Resources.

[192]  S. Berg Snowball Sampling—I , 2006 .

[193]  Claes Wohlin,et al.  An evaluation of methods for prioritizing software requirements , 1998, Inf. Softw. Technol..

[194]  Ted S. Sindlinger,et al.  Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business , 2010 .

[195]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[196]  Stephen Fickas,et al.  Goal-Directed Requirements Acquisition , 1993, Sci. Comput. Program..

[197]  Ian F. Alexander,et al.  Understanding project sociology by modeling stakeholders , 2004, IEEE Software.

[198]  Galal H. Galal-Edeen,et al.  Stakeholder identification in the requirements engineering process , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[199]  Ronald G. Ross,et al.  Principles of the business rule approach: Ronald G. Ross, Addison-Wesley Information Technology Series, February 2003, 256pp., price £30.99, ISBN 0-201-78893-4 , 2004, Int. J. Inf. Manag..

[200]  K. Goh,et al.  Universal behavior of load distribution in scale-free networks. , 2001, Physical review letters.

[201]  A. Lewis Bastian Data validation , 1970, SIGFIDET '70.

[202]  U. Brandes A faster algorithm for betweenness centrality , 2001 .

[203]  Ronald K. Mitchell,et al.  Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of who and What Really Counts , 1997 .

[204]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[205]  Brian Randell,et al.  Software engineering : report on a conference sponsored by the NATO Science Committee, Garmisch, Germany, 7th to 11th October 1968 , 1969 .

[206]  R.N. Charette,et al.  Why software fails [software failure] , 2005, IEEE Spectrum.

[207]  Roger R. Hall,et al.  Prototyping for usability of new technology , 2001, Int. J. Hum. Comput. Stud..

[208]  Ian Witten,et al.  Data Mining , 2000 .

[209]  Stewart Brand,et al.  How Buildings Learn: What Happens After They're Built , 1997 .

[210]  Soren Lauesen,et al.  Software Requirements: Styles & Techniques , 2002 .

[211]  Edward R. Dougherty,et al.  Is cross-validation valid for small-sample microarray classification? , 2004, Bioinform..

[212]  Steve McConnell,et al.  Rapid Development: Taming Wild Software Schedules , 1996 .

[213]  Jun Wang,et al.  Unifying user-based and item-based collaborative filtering approaches by similarity fusion , 2006, SIGIR.

[214]  Mehdi T. Harandi,et al.  Workshop on software specification and design , 1988, SOEN.

[215]  R. Yin Case Study Research: Design and Methods , 1984 .

[216]  Frank Moisiadis,et al.  Prioritising scenario evolution , 2000, Proceedings Fourth International Conference on Requirements Engineering. ICRE 2000. (Cat. No.98TB100219).

[217]  Sara Jones,et al.  Discovering Web Services to Specify More Complete System Requirements , 2007, CAiSE.

[218]  Ivar Jacobson,et al.  Object-Oriented Software Engineering , 1991, TOOLS.

[219]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.