Using logic model and visualization to conduct portfolio evaluation.

Portfolio evaluation is the evaluation of multiple projects with a common purpose. While logic models have been used in many ways to support evaluation, and data visualization has been used widely to present and communicate evaluation findings, adopting logic models for portfolio evaluation and using data visualization to share findings simultaneously is surprisingly limited in the literature. With the data from a sample portfolio of 209 projects which aims to improve the system of early care and education (ECE), this study illustrated how to use logic model and data visualization techniques to conduct a portfolio evaluation by answering two evaluation questions: "To what extent are the elements of a logic model (strategies, sub-strategies, activities, outcomes, and impacts) reflected in the sample portfolio?" and "Which dominant paths through the logic model were illuminated by the data visualization technique?" For the first question, the visualization technique illuminated several dominant strategies, sub-strategies, activities, and outcomes. For the second question, our visualization techniques made it convenient to identify critical paths through the logic model. Implications for both program evaluation and program planning were discussed.

[1]  Jennifer C Greene.  Logic and evaluation theory. , 2013, Evaluation and program planning.

[2]  Robin Lin Miller.  Logic models: a useful way to study theories of evaluation practice? , 2013, Evaluation and program planning.

[3]  Annie Millar,et al.  Logic models: a systems tool for performance management , 2001 .

[4]  Lisa M. Dillman.  Comparing evaluation activities across multiple theories of practice. , 2013, Evaluation and program planning.

[5]  Ralph Renger,et al.  How Using a Logic Model Refined Our Program to Ensure Success , 2009, Health promotion practice.

[6]  Deborah Helitzer,et al.  Evaluation for community-based programs: the integration of logic models and factor analysis. , 2010, Evaluation and program planning.

[7]  Joy A. Frechtling.  Logic Modeling Methods in Program Evaluation , 2007 .

[8]  John A. McLaughlin,et al.  Logic models: a tool for telling your programs performance story , 1999 .

[9]  Logic model use in developing a survey instrument for program evaluation: emergency preparedness summits for schools of nursing in Georgia. , 2007, Public health nursing.

[10]  Mark Hansen,et al.  Depicting the logic of three evaluation theories. , 2013, Evaluation and program planning.

[11]  Michael J. Schwerin,et al.  A cluster evaluation of Navy quality of life programs , 2002 .

[12]  Kendon J. Conrad,et al.  Creating and Using Logic Models: Four Perspectives , 1999 .

[13]  Riki Savaya,et al.  The Logic Model , 2005 .

[14]  Lisa R. Holliday.  Using logic model mapping to evaluate program fidelity , 2014 .

[15]  Ralph Renger,et al.  A Three-Step Approach to Teaching Logic Models , 2002 .

[16]  Anne T. Vo,et al.  Visualizing context through theory deconstruction: a content analysis of three bodies of evaluation theory literature. , 2013, Evaluation and program planning.

[17]  Zoe A. Barley,et al.  Cluster Evaluation: A Method to Strengthen Evaluation in Smaller Programs with Similar Purposes , 1993 .

[18]  Rebecca J C Luskin,et al.  Comparing the intended consequences of three theories of evaluation. , 2013, Evaluation and program planning.

[19]  J. Bucher.  Using the Logic Model for Planning and Evaluation: Examples for New Users , 2010 .

[20]  Blaine R. Worthen,et al.  Conceptual Challenges Confronting Cluster Evaluation , 1997 .

[21]  Jianping Shen,et al.  Evaluating program effects: Conceptualizing and demonstrating a typology. , 2019, Evaluation and program planning.