Guidance counselor reports of the ASSISTments college prediction model (ACPM)

Advances in the learning analytics community have created opportunities to deliver early warnings that alert teachers and instructors when a student is at risk of not meeting academic goals [6], [71]. Alert systems have also been developed for school district leaders [33] and for academic advisors in higher education [39], but other professionals in the K-12 system, namely guidance counselors, have not been widely served by these systems. In this study, we use college enrollment models created for the ASSISTments learning system [55] to develop reports that target the needs of these professionals, who often work directly with students, but usually not in classroom settings. These reports are designed to facilitate guidance counselors' efforts to help students to set long term academic and career goals. As such, they provide the calculated likelihood that a student will attend college (the ASSISTments College Prediction Model or ACPM), alongside student engagement and learning measures. Using design principles from risk communication research and student feedback theories to inform a co-design process, we developed reports that can inform guidance counselor efforts to support student achievement.

[1]  Ryan Shaun Joazeiro de Baker,et al.  Adapting to When Students Game an Intelligent Tutoring System , 2006, Intelligent Tutoring Systems.

[2]  Neil T. Heffernan,et al.  Informing Teachers Live about Student Learning: Reporting in Assistment System , 2005 .

[3]  Kasia Muldner,et al.  Exploring the Impact of a Learning Dashboard on Student Affect , 2015, AIED.

[4]  Jessica S Ancker,et al.  Rethinking health numeracy: a multidisciplinary literature review. , 2007, Journal of the American Medical Informatics Association : JAMIA.

[5]  James E. Ysseldyke,et al.  Expectancy and halo effects as a result of artificially induced teacher bias , 1976 .

[6]  J. Schaubroeck,et al.  Achievement Goals, Feedback, and Task Performance , 2010 .

[7]  Jared E Knowles,et al.  Of Needles and Haystacks: Building an Accurate Statewide Dropout Early Warning System in Wisconsin , 2015, EDM 2015.

[8]  Arnon Hershkovitz,et al.  Discovery With Models , 2013 .

[9]  Joseph C. Bentley The Educational Decision-makers. , 1964 .

[10]  Yingyi Ma,et al.  Family Socioeconomic Status, Parental Involvement, and College Major Choices—Gender, Race/Ethnic, and Nativity Patterns , 2009 .

[11]  Sirkka L. Jarvenpaa,et al.  Graphics and managerial decision making: research-based guidelines , 1988, CACM.

[12]  Carol S. Dweck,et al.  Mindsets and Math/Science Achievement , 2008 .

[13]  Laura W. Perna STUDYING COLLEGE ACCESS AND CHOICE: A PROPOSED CONCEPTUAL MODEL , 2006 .

[14]  R. Butler Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. , 1987 .

[15]  James E. Rosenbaum,et al.  Gatekeeping in an Era of More Open Gates: High School Counselors' Views of Their Influence on Students' College Plans , 1996, American Journal of Education.

[16]  Jaron Todd Holmes,et al.  Effect of Message Framing on Reactions to Feedback Messages, Moderated by Regulatory Focus , 2008 .

[17]  Norbert Schwarz,et al.  Response scales as frames of reference: The impact of frequency range on diagnostic judgements , 1991 .

[18]  I. Levin,et al.  How Consumers Are Affected by the Framing of Attribute Information Before and After Consuming the Product , 1988 .

[19]  Jonathan A. Schwabish An Economist's Guide to Visualizing Data , 2014 .

[20]  Neil T. Heffernan,et al.  Predicting STEM and Non-STEM College Major Enrollment from Middle School Interaction with Mathematics Educational Software , 2014, EDM.

[21]  Arthur C. Graesser,et al.  Responding to Learners' Cognitive-Affective States with Supportive and Shakeup Dialogues , 2009, HCI.

[22]  Rita Kukafka,et al.  Using design principles to foster understanding of complex health concepts in consumer informatics tools. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[23]  Andreas Hergovich,et al.  The impact of feedback on goal setting and task performance - Testing the feedback intervention theory , 2013 .

[24]  Beverly Park Woolf,et al.  Repairing Disengagement With Non-Invasive Interventions , 2007, AIED.

[25]  Dianne C. Berry,et al.  Effects of presenting the baseline risk when communicating absolute and relative risk reductions , 2005 .

[26]  Eric J. Johnson,et al.  Information displays and preference reversals , 1988 .

[27]  Jeremy Roschelle,et al.  Designing Formative Assessment Software with Teachers: an Analysis of the Co-Design Process , 2007, Res. Pract. Technol. Enhanc. Learn..

[28]  Zachary A. Pardos,et al.  Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes , 2013, LAK '13.

[29]  Roger L. Hutchinson,et al.  Functions of Secondary School Counselors in the Public Schools: Ideal and Actual. , 1986 .

[30]  Timothy D. Wilson,et al.  The halo effect: Evidence for unconscious alteration of judgments. , 1977 .

[31]  J. Colyar,et al.  School Counsel , 2004 .

[32]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[33]  Glenn W. Lambie,et al.  The Challenge to Change from Guidance Counseling to Professional School Counseling: A Historical Proposition , 2004 .

[34]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[35]  Neil T. Heffernan,et al.  Exploring college major choice and middle school student behavior, affect and learning: what happens to students who game the system? , 2015, LAK.

[36]  Jason H. Sharp,et al.  Semiotics as a Theoretical Foundation of Information Design , 2011 .

[37]  D. Wigboldus,et al.  From task characteristics to learning: A systematic review. , 2010, Scandinavian journal of psychology.

[38]  Barbara Tversky,et al.  Prolegomenon to Scientific Visualizations , 2005 .

[39]  Wolfgang Schnotz,et al.  Feedback Effects on Performance, Motivation and Mood: Are They Moderated by the Learner's Self-Concept? , 2014 .

[40]  Susan Gerofsky,et al.  ANCESTRAL GENRES OF MATHEMATICAL GRAPHS , 2011 .

[41]  Gary D. Sandefur,et al.  Family resources, social capital, and college attendance , 2006 .

[42]  Dylan Wiliam,et al.  The role of formative assessment in effective learning environments , 2010 .

[43]  P. Slovic,et al.  The Use of Narrative Evidence and Explicit Likelihood by Decisionmakers Varying in Numeracy , 2009, Risk analysis : an official publication of the Society for Risk Analysis.

[44]  Anita Rawls,et al.  Rules of Engagement , 2013 .

[45]  V. Shute Focus on Formative Feedback , 2007 .

[46]  Steven D. Brown,et al.  Toward a Unifying Social Cognitive Theory of Career and Academic Interest, Choice, and Performance , 1994 .

[47]  Ronald L. Partin,et al.  School Counselors' Time: Where Does It Go?. , 1993 .

[48]  Hanna Dumont and David Istance and Francisco Benavides The nature of learning : using research to inspire practice , 2010 .

[49]  Neil T. Heffernan,et al.  Population validity for educational data mining models: A case study in affect detection , 2014, Br. J. Educ. Technol..

[50]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[51]  References , 1971 .

[52]  Steven Lonn,et al.  Bridging the gap from knowledge to action: putting analytics in the hands of academic advisors , 2012, LAK '12.

[53]  Kristiina Karvonen,et al.  The beauty of simplicity , 2000, CUU '00.

[54]  David Boud,et al.  Feedback Models for Learning, Teaching and Performance , 2014 .

[55]  Karel M. Stokking,et al.  Framing of feedback impacts student’s satisfaction, self-efficacy and performance , 2015, Advances in health sciences education : theory and practice.

[56]  Neil T. Heffernan,et al.  Predicting College Enrollment from Student Interaction with an Intelligent Tutoring System in Middle School , 2013, EDM.

[57]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[58]  John A. Sparrow,et al.  Graphical displays in information systems: some data properties influencing the effectiveness of alternative forms , 1989 .

[59]  David K. Simkin,et al.  An Information-Processing Analysis of Graph Perception , 1987 .

[60]  Kasia Muldner,et al.  A Multimedia Adaptive Tutoring System for Mathematics that Addresses Cognition, Metacognition and Affect , 2014, International Journal of Artificial Intelligence in Education.

[61]  Kenneth J. Dunegan Fines, Frames, and Images: Examining Formulation Effects on Punishment Decisions , 1996 .

[62]  Neil T. Heffernan,et al.  The ASSISTment Builder: Supporting the Life Cycle of Tutoring System Content Creation , 2009, IEEE Transactions on Learning Technologies.

[63]  Neil T. Heffernan,et al.  Increasing Parent Engagement in Student Learning Using an Intelligent Tutoring System , 2011 .

[64]  P. Shah,et al.  Review of Graph Comprehension Research: Implications for Instruction , 2002 .

[65]  Stephen Michael Kosslyn,et al.  Graph Design for the Eye and Mind , 2006 .

[66]  Jeffrey B Vancouver,et al.  The effect of feedback sign on task performance depends on self-concept discrepancies. , 2004, The Journal of applied psychology.

[67]  Joy J. Burnham,et al.  School Counselor Roles: Discrepancies between Actual Practice and Existing Models. , 2000 .

[68]  Jessica S. Ancker,et al.  The Practice of Informatics: Design Features of Graphs in Health Risk Communication: A Systematic Review , 2006, J. Am. Medical Informatics Assoc..

[69]  W. Pelham,et al.  Teachers' ratings of disruptive behaviors: The influence of halo effects , 1993, Journal of abnormal child psychology.

[70]  Neil T. Heffernan,et al.  Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[71]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.