Studying teacher selection of resources in an ultra-large scale interactive system: Does metadata guide the way?

Ultra-large-scale interactive systems on the Internet have begun to change how teachers prepare for instruction, particularly in regards to resource selection. Consequently, it is important to look at how teachers are currently selecting resources beyond content or keyword search. We conducted a two-part observational study of an existing popular system called TeachersPayTeachers hypothesizing that 'evaluative metadata' (i.e. comments, ratings, and popularity measures) would drive selection of resources. The first part examined patterns in tens of thousands of sales overall, and the second part focused on patterns of sales in one focal topic that could be expert coded. We find that there are significant gaps in available metadata, that some aspects of metadata are closely associated with sales, and that metadata are weak correlates of expert-determined quality. We conclude by making suggestions for additional research and suggesting how ultra-large scale-interactive systems such as TeachersPayTeachers could be used to improve teacher education.

[1]  Jerry Li,et al.  Web-Based Tools for Learning Object Evaluation , 2004 .

[2]  Cheryl A. Franklin Factors That Influence Elementary Teachers Use of Computers , 2005 .

[3]  Ivor Goodson,et al.  Exploring the Teacher's Professional Knowledge: Constructing Identity and Community. , 1994 .

[4]  Luis E. Anido-Rifón,et al.  Creating the first SCORM object , 2008, Comput. Educ..

[5]  Mark S. Ackerman,et al.  The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility , 2000, Hum. Comput. Interact..

[6]  Mimi Recker,et al.  Understanding educator perceptions of "quality" in digital libraries , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[7]  Christine Greenhow,et al.  Learning, Teaching, and Scholarship in a Digital Age , 2009 .

[8]  John Webb,et al.  Editorial: Editor Sings the Blues , 2005 .

[9]  Edward A. Fox,et al.  "What is a good digital library?" - A quality model for digital libraries , 2007, Inf. Process. Manag..

[10]  Daniel D. Suthers Evaluating the Learning Object Metadata for K-12 educational resources , 2001, Proceedings IEEE International Conference on Advanced Learning Technologies.

[11]  Judy H. Jeng What is usability in the context of the digital library and how can it be measured , 2005 .

[12]  John C. Nesbit,et al.  A Convergent Participation Model for Evaluation of Learning Objects , 2002 .

[13]  Susan Loucks-Horsley,et al.  Five Models of Staff Development for Teachers. , 1989 .

[14]  Rachael Gabriel,et al.  A Practice-Based Theory of Professional Education , 2011 .

[15]  Annemarie Hauf,et al.  Computers in education , 1983 .

[16]  Diane I. Hillmann,et al.  The Continuum of Metadata Quality: Defining, Expressing, Exploiting , 2004 .

[17]  A. M. Madni,et al.  Recommender systems in e-commerce , 2014, 2014 World Automation Congress (WAC).

[18]  Mark S. Schlager,et al.  Building Virtual Communities: Evolution of an Online Education Community of Practice , 2002 .

[19]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[20]  S. Kiesler,et al.  Applying Common Identity and Bond Theory to Design of Online Communities , 2007 .

[21]  Ye Liu,et al.  You can lead a horse to water: teacher development and use of digital library resources , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[22]  Udo Winand,et al.  The sharable content object reference model (SCORM) - a critical review , 2002, International Conference on Computers in Education, 2002. Proceedings..

[23]  G. Watson Technology Professional Development: Long-Term Effects on Teacher Self-Efficacy , 2006 .

[24]  Donnavieve N. Smith,et al.  Online Peer and Editorial Recommendations, Trust, and Choice in Virtual Markets , 2005 .

[25]  Hei-Chia Wang,et al.  Teaching-Material Design Center: An ontology-based system for customizing reusable e-materials , 2006, Comput. Educ..

[26]  Mimi Recker,et al.  Discovery and Use of Online Learning Resources: Case Study Findings , 2004, J. Educ. Technol. Soc..

[27]  D. Ball,et al.  Reform by the Book: What Is—or Might Be—the Role of Curriculum Materials in Teacher Learning and Instructional Reform? , 1996 .

[28]  Stephen S. Standifird Reputation and e-commerce: eBay auctions and the asymmetrical impact of positive and negative ratings , 2001 .

[29]  Laura M. O’Dwyer,et al.  Examining Teacher Technology Use , 2003 .

[30]  Douglas C. Schmidt,et al.  Ultra-large-scale systems , 2006, OOPSLA '06.

[31]  Christian D. Schunn,et al.  A new metric for assessing group level participation in fluid teams , 2009, 2009 Atlanta Conference on Science and Innovation Policy.

[32]  Jerry Zhigang Li,et al.  Crossing Boundaries with Web-Based Tools for Learning Object Evaluation , 2004, ICWL.

[33]  James H. Martin,et al.  Automatically characterizing resource quality for educational digital libraries , 2009, JCDL '09.

[34]  Jo Ellen Roseman,et al.  How well do middle school science programs measure up? Findings from Project 2061's curriculum review , 2002 .

[35]  Michael J. Hannafin,et al.  Teaching and learning in digital environments: The resurgence of resource-based learning , 2001 .

[36]  Julita Vassileva,et al.  Design and evaluation of an adaptive incentive mechanism for sustained educational online communities , 2006, User Modeling and User-Adapted Interaction.

[37]  Mimi Recker,et al.  What do you recommend? Implementation and analyses of collaborative information filtering of web resources for education , 2003 .

[38]  W. Hawley,et al.  The essentials of effective professional development: A new consensus. , 1999 .

[39]  Xavier Jaén,et al.  The need for virtual information managers in education , 2007, Comput. Educ..

[40]  Luis E. Anido-Rifón,et al.  Educational metadata and brokerage for learning resources , 2002, Comput. Educ..

[41]  Alina Stasiecka,et al.  An approach to the quality and reusability of metadata specifications for e-learning objects , 2006, Online Inf. Rev..

[42]  Marlene Scardamalia,et al.  Computer Support for Knowledge-Building Communities , 1994 .

[43]  B. Schwartz,et al.  Maximizing Versus Satisficing : Happiness Is a Matter of Choice , 2002 .

[44]  S. Higgins,et al.  The role of powerful pedagogical strategies in curriculum development , 2002 .