An evaluation and selection problems of OSS-LMS packages

Abstract The evaluation and selection of inappropriate open source software in learning management system (OSS-LMS) packages adversely affect the business processes and functions of an organization. Thus, comprehensive insights into the evaluation and selection of OSS-LMS packages are presented in this paper on the basis of three directions. First, available OSS-LMSs are ascertained from published papers. Second, the criteria for evaluating OSS-LMS packages are specified.according to two aspects: the criteria are identified and established, followed by a crossover between them to highlight the gaps between the evaluation criteria for OSS-LMS packages and the selection problems. Third, the abilities of selection methods that appear fit to solve the problems of OSS-LMS packages based on the multi-criteria evaluation and selection problem are discussed to select the best OSS-LMS packages. Results indicate the following: (1) a list of active OSS-LMS packages; (2) the gaps on the evaluation criteria used for LMS and other problems (consisting of main groups with sub-criteria); (3) use of multi-attribute or multi-criteria decision-making (MADM/MCDM) techniques in the framework of the evaluation and selection of the OSS in education as recommended solutions.

[1]  Saqib Ubaid,et al.  A Weighted Usability Measure for E-learning Systems , 2011 .

[2]  Ioannis Stamelos,et al.  Knowledge based evaluation of software systems: a case study , 2000, Inf. Softw. Technol..

[3]  Ting-Yu Chen,et al.  The ELECTRE multicriteria analysis approach based on intuitionistic fuzzy sets , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[4]  Erik Duval,et al.  Bridging the gap between learning management systems and learning object repositories: exploiting learning context information , 2005, Advanced Industrial Conference on Telecommunications/Service Assurance with Partial and Intermittent Resources Conference/E-Learning on Telecommunications Workshop (AICT/SAPIR/ELETE'05).

[5]  Manuel Castro,et al.  New Learning Services: Customized and Secured Evaluation , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[6]  Chunzhi Wang,et al.  Application of innovative technologies on the e-learning system , 2011, 2011 6th International Conference on Computer Science & Education (ICCSE).

[7]  Maria Perez,et al.  A discrete-event simulation and continuous software evaluation on a systemic quality model: An oil industry case , 2005, Inf. Manag..

[8]  Zhi-qiang Ma,et al.  A New Method for Information System Selection , 2009, 2009 Second International Conference on Future Information Technology and Management Engineering.

[9]  Hans-Ludwig Hausen,et al.  A five step method for metric-based software evaluation - effective software metrication with respect to quality standards , 1993, Microprocess. Microprogramming.

[10]  Evangelos Triantaphyllou,et al.  Multi-criteria Decision Making Methods: A Comparative Study , 2000 .

[11]  Nadire Cavus.  The application of a multi-attribute decision-making algorithm to learning management systems evaluation , 2011, Br. J. Educ. Technol..

[12]  Basar Oztaysi,et al.  A decision model for information technology selection using AHP integrated TOPSIS-Grey , 2014 .

[13]  Joël Fisler,et al.  OLAT: The Swiss Open Source Learning Management System , 2010, 2010 International Conference on e-Education, e-Business, e-Management and e-Learning.

[14]  Pier Giuseppe Rossi,et al.  MAPIT: a pedagogical-relational ITS , 2011, WCIT.

[15]  Louis A. Le Blanc,et al.  A structured approach to the evaluation and selection of CASE tools , 1992, SAC '92.

[16]  Tanja Arh,et al.  Evaluation and selection of the most applicable learning management system , 2007 .

[17]  Jaime Lloret,et al.  Analysis and comparative of virtual learning environments , 2008 .

[18]  Andrea Tick.  The MILES Military English learning system pilot project , 2009, 2009 IEEE International Conference on Computational Cybernetics (ICCC).

[19]  Derek W. Keats,et al.  Addressing digital divide issues in a partially online masters programme in Africa: the NetTel@Africa experience , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[20]  B. B. Zaidan,et al.  Multi-criteria analysis for OS-EMR software selection problem: A comparative study , 2015, Decis. Support Syst..

[21]  Maria Manuela Cunha,et al.  Criteria for ERP selection using an AHP approach , 2013, 2013 8th Iberian Conference on Information Systems and Technologies (CISTI).

[22]  Manuel Castro,et al.  dotLAB: Integrating remote labs in dotLRN , 2011, 2011 IEEE Global Engineering Education Conference (EDUCON).

[23]  L. Javier García-Villalba,et al.  Solving technological isolation to build virtual learning communities , 2013, Multimedia Tools and Applications.

[24]  G. Diaz,et al.  Expanding the Boundaries of the Classroom: Implementation of Remote Laboratories for Industrial Electronics Disciplines , 2013, IEEE Industrial Electronics Magazine.

[25]  Rajendra M. Sonar,et al.  Analytic Hierarchy Process (AHP), Weighted Scoring Method (WSM), and Hybrid Knowledge Based System (HKBS) for Software Selection: A Comparative Study , 2009, 2009 Second International Conference on Emerging Trends in Engineering & Technology.

[26]  Ching-Lai Hwang,et al.  Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey , 1981, Lecture Notes in Economics and Mathematical Systems.

[27]  Wei-Wen Wu,et al.  Selecting knowledge management strategies by using the analytic network process , 2007, Expert Syst. Appl..

[28]  Rajendra M. Sonar,et al.  Framework for evaluation and selection of the software packages: A hybrid knowledge based system approach , 2011, J. Syst. Softw..

[29]  Iván Martínez-Ortiz,et al.  Deploying and debugging educational games using e-Learning standards , 2012, Proceedings of the 2012 IEEE Global Engineering Education Conference (EDUCON).

[30]  Jace Hargis,et al.  Facilitating teaching and learning: From proprietary to community source , 2012, 2012 International Conference on Information Technology Based Higher Education and Training (ITHET).

[31]  Rex B. Kline,et al.  Usability measurement and metrics: A consolidated model , 2006, Software Quality Journal.

[32]  YazganHarun Resit,et al.  An ERP software selection process with using artificial neural network based on analytic network process approach , 2009 .

[33]  L. Chu,et al.  Information technology and its role in anaesthesia training and continuing medical education. , 2012, Best practice & research. Clinical anaesthesiology.

[34]  Shahron Williams van Rooij,et al.  Open-source learning management systems: a predictive model for higher education , 2012 .

[35]  Monica Leba,et al.  QFD – Method for eLearning Systems Evaluation , 2013 .

[36]  Andrea Capiluppi,et al.  Engaging without Over-Powering: A Case Study of a FLOSS Project , 2010, OSS.

[37]  Evangelos Triantaphyllou,et al.  Development and evaluation of five fuzzy multiattribute decision-making methods , 1996, Int. J. Approx. Reason..

[38]  Hussein Zedan,et al.  Why Moodle , 2008, 2008 12th IEEE International Workshop on Future Trends of Distributed Computing Systems.

[39]  Muhammad Wannous,et al.  Supporting the delivery of learning-contents with laboratory activities in Sakai: Work-in-progress report , 2010, IEEE EDUCON 2010 Conference.

[40]  Mohsen Khademi Dehnavi,et al.  Presenting a multimodal biometric model for tracking the students in virtual classes , 2011 .

[41]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[42]  Alexander Schober,et al.  Impact factors for learner motivation in Blended Learning environments , 2012, 2012 15th International Conference on Interactive Collaborative Learning (ICL).

[43]  Shu-Ling Lin,et al.  Grey relational analysis of students' behavior in LMS , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[44]  M. Terbuc.  Use of Free/Open Source Software in e-education , 2006, 2006 12th International Power Electronics and Motion Control Conference.

[45]  Abdul Nasir Zulkifli,et al.  An investigation of knowledge creation process in the LearningZone Learning Management System amongst postgraduate students , 2012 .

[46]  Qingtang Liu,et al.  The Research on Permission Management of the Open-source Sakai Platform , 2009, 2009 International Conference on Industrial and Information Systems.

[47]  Ján Šaliga,et al.  Hardware and software platform for ADCWAN remote laboratory , 2012 .

[48]  Ting-Sheng Wen,et al.  The study of e-learning for geographic information curriculum in higher education , 2007 .

[49]  Elio Sancristobal Ruiz,et al.  Open Service-Oriented Platforms for Personal Learning Environments , 2013, IEEE Internet Computing.

[50]  Xavier Franch,et al.  Using Quality Models in Software Package Selection , 2003, IEEE Softw..

[51]  Beate List,et al.  An evaluation of open source e-learning platforms stressing adaptation issues , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[52]  Norliza Katuk,et al.  A Multiple Attribute Decision Making for Improving Information Security Control Assessment , 2014 .

[53]  R. Mohd Bekri,et al.  Development of Malaysia Skills Certificate E-portfolio: A Conceptual Framework☆ , 2013 .

[54]  María José Rodriguez-Conde,et al.  Perceived openness of Learning Management Systems by students and teachers in education and technology courses , 2014, Comput. Hum. Behav..

[55]  Jing Liu,et al.  Notice of Retraction Applied Research on Atutor , 2009 .

[56]  C. Scharff,et al.  Global perceptions on the use of WeBWorK as an online tutor for computer science , 2008, 2008 38th Annual Frontiers in Education Conference.

[57]  Evangelos Triantaphyllou,et al.  Multi-Criteria Decision Making: An Operations Research Approach , 1998 .

[58]  B. B. Zaidan,et al.  Open source EMR software: Profiling, insights and hands-on analysis , 2014, Comput. Methods Programs Biomed..

[59]  Nigel Bevan,et al.  Quality in use: Meeting user needs for quality , 1999, J. Syst. Softw..

[60]  A. A. Zaidan,et al.  Study on open source learning management systems: A survey, profile, and taxonomy , 2015 .

[61]  Eva Ocelíková,et al.  Multi-criteria decision making methods , 2005 .

[62]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[63]  Abdul Nasir Zulkifli,et al.  Effect of demographic factors on knowledge creation processes in learning management system among postgraduate students , 2011, 2011 IEEE Conference on Open Systems.

[64]  Sebastián Ventura,et al.  Predicting students' final performance from participation in on-line discussion forums , 2013, Comput. Educ..

[65]  German Gamazo,et al.  Multi-criteria decision-making in the selection of a renewable energy project in spain: The Vikor method , 2011 .

[66]  Vijayan Sugumaran,et al.  Open Source CRM Software Selection using the Analytic Hierarchy Process , 2014, Inf. Syst. Manag..

[67]  K. Nagi,et al.  Research analysis of moodle reports to gauge the level of interactivity in elearning courses at Assumption University, Thailand , 2008, 2008 International Conference on Computer and Communication Engineering.

[68]  B. Pravalpruk,et al.  LearnSquare: Thai open-source Learning Management System , 2008, 2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology.

[69]  Hamid R. Arabnia,et al.  E-Learning, E-Business, Enterprise Information Systems, and E-Government , 2015 .

[70]  Mohamed Sarrab,et al.  Empirical study of open source software selection for adoption, based on software quality characteristics , 2014, Adv. Eng. Softw..

[71]  Alenka Kavcic.  Implementing content packaging standards , 2011, 2011 IEEE EUROCON - International Conference on Computer as a Tool.

[72]  Andreas S. Andreou,et al.  A quality framework for developing and evaluating original software components , 2007, Inf. Softw. Technol..

[73]  Shahron Williams van Rooij,et al.  Higher education sub-cultures and open source adoption , 2011, Comput. Educ..

[74]  Yasaman Mohammadshahi,et al.  A state-of-art survey on TQM applications using MCDM techniques , 2013 .

[75]  Cristina Segal,et al.  Methods to evaluate open source learning platforms , 2011, 2011 IEEE Global Engineering Education Conference (EDUCON).

[76]  Xavier Franch,et al.  A quality-model-based approach for describing and evaluating software packages , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[77]  Jan Ligus,et al.  Cybernetic Education Centre: Monitoring and control of learner's e-learning study in the field of Cybernetics and Automation by Coloured Petri Nets model , 2012, 2012 International Conference on Information Technology Based Higher Education and Training (ITHET).

[78]  Nadire Cavus,et al.  The evaluation of Learning Management Systems using an artificial intelligence fuzzy logic algorithm , 2010, Adv. Eng. Softw..

[79]  Nor Badrul Anuar,et al.  Cloud Service Selection Using Multicriteria Decision Analysis , 2014, TheScientificWorldJournal.

[80]  Cansu Cigdem Aydin,et al.  Open source learning management systems in e-learning and Moodle , 2010, IEEE EDUCON 2010 Conference.

[81]  Jian-Qiang Wang,et al.  A Method to Dynamic Stochastic Multicriteria Decision Making with Log-Normally Distributed Random Variables , 2013, TheScientificWorldJournal.

[82]  Mohamad Yusof Darus,et al.  Evaluation of an Open Source Learning Management System: Claroline , 2012 .

[83]  David W. Brooks,et al.  Using audio introductions to improve programming and oral skills in CS0 students , 2013 .

[84]  Marko Bohanec,et al.  DEX: An Expert System Shell for Decision Support • , 1990 .

[85]  Juliet Stoltenkamp,et al.  E-Learning change management and communication strategies within a HEI in a developing country: Institutional organisational cultural change at the University of the Western Cape , 2011, Education and Information Technologies.

[86]  G.G.D. Nishantha,et al.  A survey on how Blackboard is assisting educational institutions around the world and the future trends , 2010, 2010 The 12th International Conference on Advanced Communication Technology (ICACT).

[87]  S. Martin,et al.  Shareable educational architectures for remote laboratories , 2012, 2012 Technologies Applied to Electronics Teaching (TAEE).

[88]  Francisco J. García-Peñalvo,et al.  Clustering Projects for eLearning Interoperability , 2012, J. Univers. Comput. Sci..

[89]  Seema Purohit,et al.  Study the applicability of Moodle in management education , 2010, 2010 International Conference on Technology for Education.

[90]  Farookh Khadeer Hussain,et al.  Iaas Cloud Selection using MCDM Methods , 2012, 2012 IEEE Ninth International Conference on e-Business Engineering.

[91]  Rajendra M. Sonar,et al.  Evaluating and selecting software packages: A review , 2009, Inf. Softw. Technol..

[92]  Eric W. T. Ngai,et al.  Evaluation of knowledge management tools using AHP , 2005, Expert Syst. Appl..

[93]  Ahmad Kamil Mahmood,et al.  Trust -Based Service Selection in Public Cloud Computing Using Fuzzy Modified VIKOR Method , 2013 .

[94]  Bojan Srdjevic,et al.  AHP Supported Evaluation of LMS Quality , 2012, I-UxSED.

[95]  Basar Öztaysi,et al.  A decision model for information technology selection using AHP integrated TOPSIS-Grey: The case of content management systems , 2014, Knowl. Based Syst..

[96]  Enrique Herrera-Viedma,et al.  A review of quality evaluation of digital libraries based on users’ perceptions , 2012, J. Inf. Sci..

[97]  W. Hardt,et al.  Efficient course creation with templates in the OPAL Learning Management System , 2012, 2012 2nd Interdisciplinary Engineering Design Education Conference (IEDEC).

[98]  Harun Resit Yazgan,et al.  An ERP software selection process with using artificial neural network based on analytic network process approach , 2009, Expert Syst. Appl..

[99]  Sung Yul Rhew,et al.  A Quality Model for Open Source Software Selection , 2007, Sixth International Conference on Advanced Language Processing and Web Information Technology (ALPIT 2007).

[100]  Wolfgang Ossadnik,et al.  AHP-based evaluation of AHP-Software , 1999, Eur. J. Oper. Res..

[101]  Edmundas Kazimieras Zavadskas,et al.  New KEMIRA Method for Determining Criteria Priority and Weights in Solving MCDM Problem , 2014, Int. J. Inf. Technol. Decis. Mak..

[102]  Ping-Yu Hsu,et al.  A fuzzy-based decision-making procedure for data warehouse system selection , 2007, Expert Syst. Appl..

[103]  Nouran Radwan.  Current Trends and Challenges of Developing and Evaluating Learning Management Systems , 2014 .

[104]  Heinz Ulrich Hoppe,et al.  Supporting Cognitive Competence Development in Virtual Classrooms - Personal Learning Management and Evaluation Using Pedagogical Agents , 2010, 2010 10th IEEE International Conference on Advanced Learning Technologies.

[105]  Borka Jerman-Blazic,et al.  A Multi-attribute Decision Support Model for Learning Management Systems Evaluation , 2007, First International Conference on the Digital Society (ICDS'07).

[106]  Hazura Mohamed,et al.  Study on open source learning management systems , 2015 .

[107]  Alexis Tsoukiàs,et al.  IusWare: a methodology for the evaluation and selection of software products , 1997, IEE Proc. Softw. Eng..

[108]  Marc Alier,et al.  Moodle 2.0: Shifting from a Learning Toolkit to a Open Learning Platform , 2010 .

[109]  George S. Ioannidis,et al.  Web-design for learning primary school Science using LMSs: Evaluating specially designed task-oriented design using young schoolchildren , 2011, 2011 14th International Conference on Interactive Collaborative Learning.

[110]  Sung Yul Rhew,et al.  A Selection Process of COTS Components Based on the Quality of Software in a Special Attention to Internet , 2003, Human.Society@Internet 2003.

[111]  B. B. Zaidan,et al.  Evaluation and selection of open-source EMR software packages based on integrated AHP and TOPSIS , 2015, J. Biomed. Informatics.

[112]  Agustín C. Caminero,et al.  Choosing the right LMS: A performance evaluation of three open-source LMS , 2013, 2013 IEEE Global Engineering Education Conference (EDUCON).