Learning Object Evaluation: Computer-Mediated Collaboration And Inter-Rater Reliability
Learning objects offer increased ability to share learning resources so that system-wide production costs can be reduced. But how can users select from a set of similar learning objects in a repository and be assured of quality? This article reviews recent developments in the establishment of learning object repositories and metadata standards, and presents a formative reliability analysis of an online, collaborative method for evaluating quality of learning objects. The method uses a 10-item Learning Object Review Instrument (LORI) within a Convergent Participation evaluation model that brings together instructional designers, media developers, and instructors. The inter-rater reliability analysis of 12 raters evaluating eight learning objects identified specific items in LORI that require further development. Overall, the collaborative process substantially increased the reliability and validity of aggregate learning object ratings. The study concludes with specific recommendations including changes to LORI items, a rater training process, and requirements for selecting an evaluation team.
A Framework for Evaluating the Quality of Multimedia Learning Resources
This article presents the structure and theoretical foundations of the Learning Object Review Instrument (LORI), an evaluation aid available through the E-Learning Research and Assessment Network at http://www.elera.net. A primary goal of LORI is to balance assessment validity with efficiency of the evaluation process. The instrument enables learning object users to create reviews consisting of ratings and comments on nine dimensions of quality: content quality, learning goal alignment, feedback and adaptation, motivation, presentation design, interaction usability, accessibility, reusability, and standards compliance. The article presents research and practices relevant to these dimensions and describes how each dimension can be interpreted to evaluate multimedia learning resources.
genetic algorithm positioning system process control sample size solar cell visible light dna sequence learning object indoor positioning received signal strength statistical process control indoor localization quantum dot statistical proces indoor positioning system count datum hecke algebra factorial design ieee standard binding site escherichia coli weighted moving average knowledge structure statistical quality control poisson structure cell cycle choice behavior econometric model quality level exponentially weighted moving fractional factorial design saccharomyces cerevisiae selection bia affine weyl group statistical process monitoring power conversion efficiency dye-sensitized solar cell charge transport uniform resource identifier learning object metadatum embryonic stem cell moving average control object class dye-sensitized solar reusable learning object linkage disequilibrium quantity discount spatial process spatial econometric population parameter embryonic stem reusable learning object metadatum heterojunction solar cell dna repair location fingerprinting cell development indoor positioning technique spatial econometric model radiation tolerance heterojunction solar genetic linkage signal peptide bulk heterojunction dna segment recombination rate bulk heterojunction solar dna recombination wifi-based indoor localization surface recombination escherichia coli. low-density lipoprotein indoor positioning solution proposed positioning system surface recombination velocity solar cells. neisseria meningitidi genetic heterogeneity learning object review dna break xrcc5 wt allele xrcc5 gene t cell receptor v(d)j recombination v(d)j recombination-activating protein 1 excretory function neuritis, autoimmune, experimental leukemia, b-cell dna sequence rearrangement immunoglobulin class switch recombination immunoglobulin class switching lipoprotein receptor dna breaks, double-stranded telomere maintenance v(d)j recombination genome encoded entity vdj recombinase recombination, genetic crossover (genetic algorithm) meiotic recombination homologous recombination