Are Handheld Computers Dependable? A New Data Collection System for Classroom-Based Observations

Very little research exists on the dependability of handheld computers used in public school classrooms. This study addresses four dependability criteria—reliability, maintainability, availability, and safety—to evaluate a data collection tool on a handheld computer. Data were collected from five sources: (1) time-use estimations by 19 special education teachers using handheld computers, (2) observations of time use from eight external data collectors, (3) teacher interviews, (4) technical reports, and (5) teacher satisfaction surveys. Results indicate that this data collection tool satisfied all four dependability criteria. The study also confirms that dependable handheld computers can replace paper-based data collection in educational settings.

[1]  A LOW‐TECH METHOD FOR CONDUCTING REAL‐TIME RECORDING , 1999 .

[2]  Darin J Erickson,et al.  Handheld Computers , 2003, Evaluation review.

[3]  Stephen E. Lapinsky,et al.  Review Paper: Handheld Computing in Medicine , 2003, J. Am. Medical Informatics Assoc..

[4]  Heinz Ulrich Hoppe,et al.  Guest editorial: Wireless and Mobile Technologies in Education , 2003, J. Comput. Assist. Learn..

[5]  Constantine Stephanidis,et al.  User Interface Design for PDAs: Lessons and Experience with the WARD-IN-HAND Prototype , 2002, User Interfaces for All.

[6]  Clay Spinuzzi.  Using a handheld PC to collect and analyze observational data , 2003, SIGDOC '03.

[7]  D. Fisher.  The Use of Instructional Time in the Typical High School Classroom , 2009 .

[8]  G. F. Hayhoe.  From desktop to palmtop: creating usable online documents for wireless and handheld devices , 2001, IPCC 2001. Communication Dimensions. Proceedings IEEE International Professional Communication Conference (Cat. No.01CH37271).

[9]  Melissa Roderick,et al.  Extended Learning Time and Student Accountability: Assessing Outcomes and Options for Elementary and Middle Grades , 2005 .

[10]  Roy Sterritt,et al.  Autonomic Computing - a means of achieving dependability? , 2003, 10th IEEE International Conference and Workshop on the Engineering of Computer-Based Systems, 2003. Proceedings..

[11]  N Heddle,et al.  Comparing hand‐held computers and paper diaries for haemophilia home therapy: a randomized trial , 2004, Haemophilia : the official journal of the World Federation of Hemophilia.

[12]  Gillian H. Ice.  Technological Advances in Observational Data Collection: The Advantages and LImitations of Computer-Assisted Data Collection , 2004 .

[13]  Timothy J. Lewis,et al.  Assessing Teacher Use of Opportunities to Respond and Effective Classroom Management Strategies , 2009 .

[14]  C. Weston,et al.  Debriefing interview and Colloquium: How effective are these as research strategies? , 2002 .

[15]  S. Kahng,et al.  COMPUTERIZED SYSTEMS FOR COLLECTING REAL‐TIME OBSERVATIONAL DATA , 1998 .

[16]  J. Stichter,et al.  Examining the Influence of Teacher Behavior and Classroom Context on the Behavioral and Academic Outcomes for Students With Emotional or Behavioral Disorders , 2008 .

[17]  Dave L. Edyburn.  Critical issues in special education technology research: What do we know? What do we need to know? , 2001 .

[18]  Todd K. Malan,et al.  HAND‐HELD COMPUTER OPERATING SYSTEM PROGRAM FOR COLLECTION OF RESIDENT EXPERIENCE DATA , 2000, Obstetrics and gynecology.

[19]  Clarence C. Gravlee,et al.  Mobile Computer-Assisted Personal Interviewing with Handheld Computers: The Entryware System 3.0 , 2002 .

[20]  Dorene D. Ross,et al.  Critical Features of Special Education Teacher Preparation , 2005 .

[21]  V. Anfara,et al.  Qualitative Analysis on Stage: Making the Research Process More Public , 2002 .

[22]  Edith de Leeuw,et al.  Technological Innovations in Data Collection: Acceptance, Data Quality and Costs , 1996 .

[23]  C L Berthelsen,et al.  Automated personal health inventory for dentistry: a pilot study. , 2000, Journal of the American Dental Association.

[24]  Ralph Stacey,et al.  The Emergence of Knowledge in Organization , 2000 .

[25]  David R. Morse,et al.  Using while moving: HCI issues in fieldwork environments , 2000, TCHI.

[26]  Mark R Dixon.  Creating a portable data-collection system with Microsoft Embedded Visual Tools for the Pocket PC. , 2003, Journal of applied behavior analysis.

[27]  Charles R. Greenwood,et al.  Development and validation of standard classroom observation systems for school practitioners: Ecobehavioral Assessment Systems Software (EBASS) , 1994 .

[28]  Dorian G. W. Smith,et al.  Palm computer demonstrates a fast and accurate means of burn data collection. , 2000, The Journal of burn care & rehabilitation.

[29]  Kevin B. Johnson,et al.  An Electronic Menstrual Cycle Calendar: Comparison of Data Quality With a Paper Version , 2000, Menopause.

[30]  Erika S. Trapl,et al.  Use of audio-enhanced personal digital assistants for school-based data collection. , 2005, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[31]  Sarah M. Davis,et al.  Research to industry: four years of observations in classrooms using a network of handheld devices , 2002, Proceedings. IEEE International Workshop on Wireless and Mobile Technologies in Education.

[32]  Louis H. Berry.  Cognitive Effects of Web Page Design , 2000 .

[33]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[34]  Nancy L. Leech,et al.  Linking Research Questions to Mixed Methods Data Analysis Procedures 1 , 2006 .

[35]  Amihai Motro,et al.  Imprecision and incompleteness in relational databases: survey , 1990 .

[36]  A. Onwuegbuzie,et al.  Mixed Methods Research: A Research Paradigm Whose Time Has Come , 2004 .

[37]  Elliot Soloway,et al.  Pocket PiCoMap: a case study in designing and assessing a handheld concept mapping tool for learners , 2003, CHI '03.

[38]  P. Sindelar,et al.  Models and Measures of Beginning Teacher Quality , 2006 .

[39]  Gustav Mikkelsen,et al.  Consequences of impaired data quality on information retrieval in electronic patient records , 2005, Int. J. Medical Informatics.

[40]  Wilson D. Pace,et al.  Electronic Data Collection Options for Practice-Based Research Networks , 2005, The Annals of Family Medicine.

[41]  Clarence C. Gravlee,et al.  Handheld Computers for Direct Observation of the Social and Physical Environment , 2006 .

[42]  Arup Dutta,et al.  Development and use of behavior and social interaction software installed on Palm handheld for observation of a child’s social interactions with the environment , 2006, Behavior research methods.