Algorithm-Generated Versus Human-Generated Academic Plans: Determining Optimality from Community College Articulation Agreements

Our study examined one pain point users may have with community college articulation agreements: cross-referencing multiple articulation agreement reports to manually develop an optimal academic plan. Optimal is defined as the minimal set of community college courses that satisfy the transfer requirements for multiple universities a student is preparing to apply to. We recruited 24 California community college transfer students to participate in a research session that consisted of an experiment, survey, and interview. We developed a low-fidelity prototype of a report that contains an algorithmically-generated optimal academic plan. We experimentally compared the prototype to ASSIST, California's official statewide database of articulation agreement reports. Compared to students who used the prototype, students assigned to use ASSIST reports to manually create an optimal academic plan underperformed in optimality mistakes, time required, and usability scores. Moving to our non-experimental results, a sizable minority of students had a negative assessment of counselors' ability and willingness to manually create optimal academic plans using ASSIST. Our last results revolved around students' recommendations for supplemental software features to improve the optimization prototype.

[1]  Gerardo Patriotta,et al.  Why do users trust algorithms? A review and conceptualization of initial trust and trust over time , 2022, European Management Journal.

[2]  Rachel Worsham.  Understanding the landscape of community college transfer in North Carolina , 2022, New Directions for Community Colleges.

[3]  Lauren Schudde,et al.  Getting to the Core of Credit Transfer: How Do Pre-Transfer Core Credits Predict Baccalaureate Attainment for Community College Transfer Students? , 2022, Educational Policy.

[4]  David P. Reeping,et al.  Information asymmetries in web‐based information for engineering transfer students , 2021, Journal of Engineering Education.

[5]  Andrea Desantis,et al.  Early Effects of North Carolina’s Comprehensive Articulation Agreement on Credit Accumulation Among Community College Transfer Students , 2021, Research in Higher Education.

[6]  Huriya Jabbar,et al.  Thinking Through Transfer: Examining How Community College Students Make Transfer Decisions , 2020 .

[7]  Majid Shakhsi‐Niaei,et al.  An optimization‐based decision support system for students' personalized long‐term course planning , 2020, Comput. Appl. Eng. Educ..

[8]  M. Mejia,et al.  Increasing Community College Transfers: Progress and Barriers. , 2020 .

[9]  David A. Joyner,et al.  SAGA: Curricula Optimization , 2020, L@S.

[10]  S. Hayes,et al.  The Role of Academic Advisors in the Development of Transfer Student Capital , 2020, NACADA Journal.

[11]  Zachary A. Pardos,et al.  Data-Assistive Course-to-Course Articulation Using Machine Translation , 2019, L@S.

[12]  Walter C. Lee,et al.  Unnecessarily Complicated: An Examination of Information Asymmetry in the Transfer Process , 2019 .

[13]  Vijay K. Mago,et al.  Automating Articulation: Applying Natural Language Processing to Post-Secondary Credit Transfer , 2019, IEEE Access.

[14]  Matt S. Giani.  The Correlates of Credit Loss: How Demographics, Pre-Transfer Academics, and Institutions Relate to the Loss of Credits for Vertical Transfer Students , 2019, Research in Higher Education.

[15]  Z. Taylor.  Inarticulate Transfer: Do Community College Students Understand Articulation Agreements? , 2019 .

[16]  C. Kivunja.  Distinguishing between Theory, Theoretical Framework, and Conceptual Framework: A Systematic Review of Lessons from the Field , 2018, International Journal of Higher Education.

[17]  Veronika Alexander,et al.  Why trust an algorithm? Performance, cognition, and neurophysiology , 2018, Comput. Hum. Behav..

[18]  George Spencer.  Can Transfer Guides Improve the Uptake of Major Prerequisites? Evidence from Ohio’s Transfer and Articulation Policy Reform , 2018, Research in Higher Education.

[19]  Davis Jenkins,et al.  Using Data Mining to Explore Why Community College Transfer Students Earn Bachelor’s Degrees With Excess Credits , 2018 .

[20]  J. Sim,et al.  Saturation in qualitative research: exploring its conceptualization and operationalization , 2017, Quality & Quantity.

[21]  Christopher Mazzeo,et al.  Exploring Credit Mobility and Major-Specific Pathways: A Policy Analysis and Student Perspective on Community College to University Transfer , 2017 .

[22]  A. Soliz,et al.  Statewide Transfer Policies and Community College Student Success , 2017, Education Finance and Policy.

[23]  Eric J. Lichtenberger,et al.  The Community College Penalty? Examining the Bachelor’s Completion Rates of Community College Transfer Students as a Function of Time , 2017 .

[24]  Mary G. Visher,et al.  Scaling Academic Planning in Community College: A Randomized Controlled Trial. REL 2017-204. , 2016 .

[25]  Nathaniel J. Bray,et al.  Alabama Articulation and General Studies Committee & Statewide Transfer and Articulation Reporting System: Evaluation Project Final Report , 2016 .

[26]  A. Mohamed.  Interactive Decision Support for Academic Advising. , 2016 .

[27]  J. Stern.  The effect of articulation agreements on community college transfers and bachelor’s degree attainment , 2016 .

[28]  Robin R. Lasota,et al.  What Matters in Increasing Community College Students’ Upward Transfer to the Baccalaureate Degree: Findings from the Beginning Postsecondary Study 2003–2009 , 2016 .

[29]  Jie Xu,et al.  Personalized Course Sequence Recommendations , 2015, IEEE Transactions on Signal Processing.

[30]  M. Zeidenberg.  Valuable Learning or “Spinning Their Wheels”? Understanding Excess Credits Earned by Community College Associate Degree Completers , 2015 .

[31]  Benjamin L. Castleman,et al.  The Shapeless River: Does a Lack of Structure Inhibit Students’ Progress at Community Colleges? , 2015 .

[32]  B. Cooper,et al.  Transfer Credit Evaluations: How They are Produced, Why It Matters, and How to Serve Students Better. , 2014 .

[33]  M. Tavakol,et al.  Making sense of Cronbach's alpha , 2011, International journal of medical education.

[34]  Amelito G Enriquez,et al.  The Dismantling of the Engineering Education Pipeline , 2011 .

[35]  Tom Page,et al.  Feature creep and usability in consumer electronic product design , 2009 .

[36]  Xin Wang,et al.  Web ecology: Information needs of different user groups in the context of a community college website , 2009, ASIST.

[37]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[38]  Josipa Roksa,et al.  Credits, Time, and Attainment: Articulation Policies and Success After Transfer , 2008 .

[39]  B. Elliott,et al.  Anything is possible: Managing feature creep in an innovation rich environment , 2007, 2007 IEEE International Engineering Management Conference.

[40]  M. Alfonso,et al.  Effectiveness of Statewide Articulation Agreements on the Probability of Transfer: A Preliminary Policy Analysis , 2006 .

[41]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[42]  Manfred Tscheligi,et al.  Paper prototyping - what is it good for?: a comparison of paper- and computer-based low-fidelity prototyping , 2003, CHI Extended Abstracts.

[43]  James A. Landay,et al.  High-Fidelity or Low-Fidelity, Paper or Computer? Choosing Attributes when Testing Web Prototypes , 2002 .

[44]  Donna Taggart,et al.  Analysis of ASSIST Navigability, Usability, and Content Using Student Focus Groups, Spring 2000. Final Report. , 2000 .

[45]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[46]  Uzair Iqbal Janjua,et al.  The Impact of Scope Creep on Project Success: An Empirical Investigation , 2020, IEEE Access.

[47]  Eva Schiorring,et al.  Career and Technical Education (CTE) Transfer Research Project: Improving Transfer Pathways for California Community College Students in CTE Programs. , 2011 .

[48]  Soundar Kumara,et al.  ADVISOR—An expert system for the selection of courses☆ , 1992 .