“Pathways for Change: 6 Theories about How Policy Change Happens”
(download)
Developed with generous support from The California Endowment.
Advocates and funders are increasingly asked to consider the logic, or theory, behind their strategies and approaches to help describe and evaluate their efforts. This brief summarizes descriptions and underlying assumptions for six theories from different disciplines that apply to advocacy and policy change efforts. Theories covered include: Punctuated Equilibrium Theory, Advocacy Coalition Framework, Agenda Setting, Messaging and Frameworks, Power Elites and Community Organizing. Knowing about and incorporating existing social science theories into advocacy work can help advocates, funders and evaluators better communicate about their work, develop partnerships, and create meaningful evaluation plans.
“Getting Started: A Self-Directed Guide to Outcome Map Development”
Organizational Research Services (ORS) has been providing ongoing evaluation consultation to The Annie E. Casey Foundation KIDS COUNT Initiative since 2007. The focus of evaluation support has been capacity-building related to development of theory of change, identifying interim outcomes connected with advocacy and policy change efforts, developing data collection processes and tools, and using data to strengthen advocacy efforts. With support from Casey, ORS has recently developed a new tool, Getting Started: A Self-Directed Guide to Outcome Map Development. This guide, a resource for KIDS COUNT grantees as well as other advocacy organizations, outlines a series of steps that facilitate development of an outcome map – that is, a graphic expression of theory of change. Getting Started is “how to” and easy-to-use resource for advocacy organizations seeking to develop and use a theory of change outcome map to simply articulate their work for a variety of audiences – e.g. Board members, staff, funders, constituents, donors, partners, or other stakeholders - and to help advocates think about how and what to evaluate. Getting Started offers step-by-step guidance along with concrete tools, templates and examples to help advocacy organizations express their theory of change via an outcome map both to enhance communication and serve as a framework for evaluation planning.
Getting Started: A Self-Directed Guide to Outcome Map Development
GUIDE, EXERCISES and EXAMPLES
(download)
Word Version of Exercises
Getting Started Part Two Step 1 and 2 Table
(download)
Getting Started Part Two Step 3 Table
(download)
Getting Started Part Three Step 4 So That Chain Exercise
(download)
“Ten Considerations for Advocacy Evaluation Planning: Lessons Learned from KIDS COUNT Grantee Experiences”
(download)
Prepared for the Annie E. Casey Foundation, 2009
For most groups involved in advocacy and policy change work, the planning steps involved in the development and implementation of evaluation include:
- Discussion about primary purposes and needs for evaluation, including how evaluation can help to inform, support, and sustain an organization's ongoing advocacy efforts;
- Clarification and prioritization of interim measures through the creation of a theory of change, "outcome map" or pathway of change; and
- Selection of data collection approaches and tools that could best support evaluation of priority measures.
This guidance piece focuses on what it takes for advocates to address steps identified above and begin to integrate evaluation into their every-day efforts. Evaluation is new territory for many advocacy organizations. Based on our coaching with KIDS COUNT grantees who've entered that territory, and we've found that a bit of orientation could be helpful. The guide describes evaluation planning steps and offers ten considerations regarding some tricky places advocates may encounter while engaging in these steps. While tricky places might not be totally avoidable, the guide is intended to present lessons gained from KIDS COUNT grantee experiences and help prepare advocacy organizations who take steps towards evaluation of their efforts. In addition, the guide highlights the benefits of evaluative thinking and offer useful guidance for advocates' evaluation pursuits.
“Trying Out a Real Time Advocacy Self-Evaluation Process: Guidance and Considerations for KIDS COUNT Grantees”
(download)
Prepared for the Annie E. Casey Foundation, 2009
In the face of growing interest and focus on advocacy evaluation, many advocates are interested in identifying simple, useable evaluation tools that they can implement themselves without a heavy resource burden. In particular, KIDS COUNT grantees have expressed interest in a practical tool to use during intense efforts that would capture data about short-term incremental gains in an effective and meaningful way. This guidance piece, created in partnership with KIDS COUNT grantees, describes how ORS approached advocates' request, and presents considerations and guidance on how advocates could effectively try out a real time self evaluation approach to document and support their work. A focus in on how advocates might adopt intentional evaluative thinking as an integrated part of advocacy and policy change work.
“Compendium of Measurement Tools for Prevention of Child Abuse & Neglect”
(download)
Organizational Research Services has compiled a Compendium of Measurement Tools for Prevention of Child Abuse & Neglect in order to support evaluation capacity-building particularly among organizations and programs working in the area of family support and the prevention of child abuse and neglect. The Compendium is based on over ten years of coaching and support that ORS has provided to grantees of the Council for Children and Families (CCF) in Washington State and, more recently, to the Hawaii Children's Trust Fund. ORS works with grantees to select appropriate parent- and family-level outcomes, with the next step being identification, adaptation or creation of data collection tools to measure these outcomes.
This Compendium can be used by both funders and programs interested in outcome-based evaluation. In addition to tools, included in the Compendium is: 1) a summary of “lessons learned” for grant makers interested in the evaluation capacity-building approach, as well as 2) an excerpt from Outcomes for Success! describing data collection methods that may be particularly useful for community-based programs.
The Compendium consists of examples of tools developed and used by a diverse array of family support and child abuse prevention programs. Some of the tools are nationally recognized and based on formal research, while others have been custom-designed to meet the unique needs of community-based programs and participants. The tools are categorized by four protective factors correlated to the prevention of child abuse and neglect:
- Improved Social Connections
- Knowledge of Parenting & Child Development
- Nurturing & Attachment
- Parental Resilience
ORS plans to periodically update this resource as other relevant tools are identified and developed.
“Orientation to Theory of Change”
(download)
ORS developed an introduction to theory of change to help familiarize our clients with the basics of theory of change and how it fits into outcome-based planning and evaluation. This brochure includes an easy-to- follow overview of theory of change techniques, types of change, and the "Layer Cake" model.
“A Guide to Measuring Advocacy and Policy” prepared for the Annie E. Casey Foundation
(download)
As more foundations and nonprofits have engaged in advocacy and policy work to address public issues and effect social change, there is a growing desire to gauge the impact of investments in this area. How to evaluate the effectiveness of advocacy and policy work is an emerging question of interest within the philanthropic and nonprofit audiences. Answering that question, however, has proven difficult because relatively few instructive resources exist to help those who wish to measure progress in this area.
The Annie E. Casey Foundation commissioned this guide to help determine meaningful ways to measure and evaluate the impact of its advocacy and public policy grantmaking. Although a primary intent of this guide is to contribute practical guidance to the Annie E. Casey Foundation regarding evaluation of the Foundation’s investments, this guide also serves as a broad call to grantmakers to build and advance the field of evaluation in this area.
The overall purpose of this guide is twofold. To help grantmakers think about and talk about measurement of advocacy and policy, this guide puts forth a framework for naming outcomes associated with advocacy and policy as well as directions for evaluation design. The framework is intended to provide a common way to identify and talk about outcomes, providing philanthropic and nonprofit audiences an opportunity to react to, refine and adopt the outcome categories presented. In addition, grantmakers can consider some key directions for evaluation design that include a broad range of methodologies, intensities, timeframes and purposes.
As a companion to this guide, ORS has collected examples of measurement tools that are applicable to advocacy and policy work. These are presented in
“A Handbook of Data Collection Tools”: Companion to “A Guide to Measuring Advocacy and Policy.”
(download)
This handbook of data collection tools is intended to serve as a companion to “A Guide to Measuring Advocacy and Policy.” Organizational Research Services (ORS) developed this guide on behalf of the Annie E. Casey Foundation to support efforts to develop and implement an evaluation of advocacy and policy work. The companion handbook is dedicated to providing examples of practical tools and processes for collecting useful information from policy and advocacy efforts. It is available in both document form and as an online resource. Please see the
Annie E. Casey Foundation web site, ORS web site
and
Innonet web site.
Annie E. Casey Foundation Manuals
(the following manuals were commissioned for the Annie E. Casey Foundation for the Making Connections initiative):
“A Practical Guide to Documenting Influence and Leverage in Making Connections Communities”
(download)
“A Practical Guide to Documenting Influence and Leverage in Making Connections Communities” offers practical guidance to capturing and documenting influences and leverage in a Making Connections community. It provides clarification,concrete examples and suggested approaches for documentation of these often elusive concepts. The guide is intended to be versatile to meet a wide variety of specific audiences and needs.
“Theory of Change: A Practical Tool For Action, Results and Learning”
(download)
This guide describes an approach and practical strategies for engaging in theory of change work with community partners. It offers instructions in using specific techniques or “so that” chains and outcome maps in the development of a theory of change.
Organizational Research Services is the parent company of The Evaluation Forum (TEF). TEF has written a number of publications, two of which are highlighted below.
“How to Manage and Analyze Data for Outcome-based Evaluation”
| |
Section I
|
Data Collection and Data Management Systems
(download)
|
| |
Section II
|
Using Access for Data Entry
(download)
|
| |
Section III
|
Using Excel for Data Analysis
(download)
|
| |
Section IV
|
Analyzing Data
(download)
|
This handbook is an introduction to the process of preparing and analyzing data for outcome-based evaluation. It covers a range of topics to help you effectively collect and manage outcome data, and use common software packages such as Microsoft Access and Microsoft Excel to analyze your data. The handbook further provides some background material on the computation and interpretation of basic statistical measures, and how to summarize, interpret and present the results of outcome evaluations.
“Outcomes for Success!”
(order a hard copy)
“Outcomes for Success!” can be used as a stand-alone “travel guide” to help agencies and organizations on their outcomes journey. It also can be used as part of a training course provided by outcome measurement consultants. It is written for program staff, funders, policy makers and others who wish to learn about outcome planning and evaluation in an easy-to-digest, jargon-free format.
In addition, ORS has a number of publications that are
customized. Please refer
to our
Customized Manuals / Guides >> Featured Projects
section of this website to peruse other examples as well.
|
