New & Noteworthy

ORS provides consultation for Medina Foundation Evaluation Grants.
In celebration of its 60th anniversary, the Medina Foundation provided special Evaluation Grants to nonprofit organizations across the greater Puget Sound area, intended to strengthen organizations' evaluation capacity and to provide the opportunity to learn from evaluation. ORS provided consultation and guidance to the Medina Foundation in the development and implementation of these grants, including the development of two brief publications. The first, called "Guidance in Approaching Evaluation" <download>, was developed in spring 2007 as a resource to the Foundation and to prospective grantees. The second is a summary report of key learnings that emerged as a result of these innovative grants, called "Summary Report of the Medina Foundation Evaluation Grants" <download>. It is hoped that these two documents, made possible by the Medina Foundation' leadership, will be informative to the philanthropic and nonprofit fields.

Four sessions by ORS presented at American Evaluation Association 2008 Conference in Denver.
This November, at the American Evaluation Association's Annual Conference, ORS associates were pleased to present four sessions. In "Building Evidence, Building Capacity: Compatibilities and Conflicts," ORS associates drew upon their years of experience to share examples of different approaches to building both capacity and evidence, with discussion of the implications for grantmaking and evaluation. The session "Practical Guidance and Tools for Advocacy Evaluation" discussed the growing field of the evaluation of advocacy efforts, and will share examples of practical tools that are in use or development across the nation. "Evaluating Systems Change Efforts" addressed the challenges of evaluating large, complex systems such as in health and education. Finally, "Community Organizing Evaluation: Crossing the Next Frontier" provided an overview of community organizing evaluation resources, tools, and new models. Read more about the AEA Conference here.

Spotlight on ORS Assistants.
As ORS grows, three new Assistants have joined the ORS team since November 2007. Meet Emilee, Kanna, and Steve on the Meet Our Team page.

Philanthropic and nonprofit organizations benefit from Dashboards to gauge their progress.
In 2008, ORS has worked with several clients, including The Seattle Foundation and The Hawai'i Children's Trust Fund, to support the development of Dashboards. The term "Dashboard" emerged out of the corporate world, and it alludes to the idea that you can look at the dashboard of your car and see key information in measures like the gas gauge, odometer, speedometer, thermometer, etc. A Dashboard functions the same way for an organization. It works as an at-a-glance report for those who have a stake in knowing the overall shape of the organization on a regular basis, without needing to examine all the inner workings "under the hood." To learn more about how ORS can support the development of a Dashboard for your organization, contact us.

ORS launches evaluation of advocacy training workshops for librarians.
The Public Library Association developed Turning the Page: Building Your Library Community, an advocacy training event that equips librarians and library supporters with the skills, confidence, and resources they need to create community partnerships, build alliances with local and regional decision makers, and ultimately increase funding for their libraries. Turning the Page was designed specifically for library systems eligible to participate in the Opportunity Online hardware grant. These advocacy efforts in turn will help libraries sustain high-quality technology services so that all people have equal access to opportunity, information, and learning. ORS is evaluating the impact of this training, focusing on both short-term changes (i.e., changes in knowledge, skills and confidence) and long-term changes in the communities where the library systems are situated.

New "Orientation to Theory of Change" brochure is now available. ORS developed an introduction to theory of change to help familiarize our clients with the basics of theory of change and how it fits into outcome-based planning and evaluation. This brochure includes an easy to follow overview of theory of change techniques, types of change, and the "Layer Cake" model. (download brochure)

ORS creates "A Guide to Measuring Advocacy and Policy." Commissioned by the Annie E. Casey Foundation , ORS developed this guide to help determine meaningful ways to measure and evaluate the impact of its advocacy and public policy grantmaking. The framework is intended to provide a common way to identify and talk about outcomes, providing philanthropic and nonprofit audiences an opportunity to react to, refine and adopt the outcome categories presented. Read more or download the guide. An excerpt from "A Guide to Measuring Advocacy and Policy" was also published in Harvard's Spring 2007 edition of The Evaluation Exchange.

ORS provides consultation to Hawai'i Children's Trust Fund. ORS is providing consultation to the Hawai'i Children's Trust Fund (HCTF), a public-private partnership focused on the prevention of child abuse and neglect in the state of Hawaii. ORS is working with HCTF to articulate a theory of change that expresses elements of HCTF's multi-year strategies, identifies focus outcomes and serves to ground the HCTF partnership's future evaluation efforts so that HCTF can understand the results of the partnership's work. Consultation has focused helping partners find common ground, identify the priority outcomes that will guide HCTF's work for the next three to five years, identify the broad strategies that will advance the outcomes and articulate the roles of partners in contributing to prioritized outcomes. Additionally, to help strengthen the HCTF partnership's capacity-building efforts, ORS supports HCTF grantees and grantseekers with coaching and training directed at helping programs to identify and measure program outcomes.

ORS facilitates development of Kids Matter: Improving Outcomes for Children in Washington State. Kids Matter is a collaborative and comprehensive strategic framework for building the early childhood system in Washington State to improve outcomes for children. Its goal areas are access to health insurance and medical homes, mental health and social-emotional development, early care and education/child care, parenting information and support, and family support. ORS facilitated the development of an Outcome-based strategic plan for this system-building initiative as well as the development of an evaluation framework for focusing on accountability and documentation of the plan's progress. This work entailed building upon existing strategic plans and needs assessments developed by early childhood partners, facilitating external community feedback processes, facilitating internal stakeholder planning meetings and preparing user-friendly publications. ORS, on behalf of the Kids Matter early childhood system-building partners, has conducted three annual Kids Matter stakeholder surveys to document awareness and utilization of the Kids Matter framework among early childhood stakeholders and guide Kids Matter efforts. Download the Kids Matter Executive Summary.

The Brainerd Foundation engages with ORS to evaluate its strategic plan. The Brainerd Foundation, a Seattle-based conservation foundation, engaged with ORS to evaluate its strategic plan in the context of its grantmaking priorities and practices. The goals of the process were to identify the successes of the Foundation's current strategic plan and to identify new strategic directions based on an assessment of their grantmaking accomplishments and learnings as well as foundation practices. To achieve these goals, ORS consulted with staff, trustees and convened a stakeholder partnership in order to articulate a Foundation logic model that corresponded with its strategic imperatives. We further documented accomplishments and learnings of past grantmaking and carried out surveys and interviews. Our report to the Foundation was designed as an objective assessment of the degree to which its existing strategic imperatives were being met and a practical guidance for stimulating work and revisions in its ongoing strategic planning about grantmaking and other Foundation involvements. ORS is currently working with the Brainerd Foundation to refine their grantmaking and evaluation processes to fit with their new strategic directions.

ORS conducts evaluation of City of Seattle Parent-Child Home Program's Play & Learn Demonstration Project for the Business Partnership for Early Learning. The Business Partnership for Early Learning (BPEL) is a coalition of King County business leaders committed to investing resources to promote early learning to ensure all children can reach their full potential in the classroom. BPEL's strategy for helping this population of children be ready for school, in partnership with the City of Seattle, includes funding, deploying and testing a research-based approach that helps families help their children be ready for school: Parent-Child Home Program (PCHP) and Play & Learn Groups. Locally-funded agencies target services for those children most likely to have a school preparedness gap: children who live in poverty and are either African-American or Native American or part of a family that does not speak English at home. ORS conducts outcome and process evaluation of this two-year program which aims to improve the quality of parent-child interaction, increase positive parenting behaviors and increase children's skills in order to ensure that children are more likely to enter school prepared and experience academic success. ORS has identified data collection processes and valid, reliable tools that can effectively and respectfully measure outcomes with a diverse population of families, including families who speak little or no English. A key aspect of the evaluation is to support local agencies, many of whose employees have diverse language and cultural backgrounds, in the implementation of the evaluation design by providing training, coaching and regular communication about evaluation activities. In addition, ORS conducts data analysis annually and reports program results.

ORS provides coaching to investees of Social Venture Partners – Seattle. Social Venture Partners is a philanthropic giving circle that invests in the capacity of nonprofit agencies and the growth and development of philanthropy. As part of SVP's capacity-building strategy, ORS provides one-on-one coaching and technical assistance to local investees (grantees) in enhancing their capacity to design and implement Outcome-based program evaluation. ORS staff works over a three- to five-year period with each investee to ensure that they have designed an evaluation system that fits within the context of their nonprofit agency and that they have the skills and infrastructure to carry out the evaluation. This coaching and technical assistance spans all aspects of evaluation, depending on the specific level of development and capacity of each program and agency. These aspects include: theory of change design, logic model development, outcome and indicator selection, data collection tool selection and design, development of an evaluation plan, data collection procedures, data management, data analysis, and using data for program improvement, accountability, marketing and strategic planning. We have worked with more than 30 programs on behalf of SVP and continue to enjoy beginning new coaching relationships with investees that SVP funds.

ORS provides evaluation technical assistance and training for Centers for Sexual Assault Prevention, Office of Crime Victims Advocacy. The Office of Crime Victims Advocacy (OCVA) manages policy and program development for the Sexual Assault Prevention Program and administers funding to local communities in Washington state for sexual assault prevention activities. Funds are provided to local Centers for Sexual Assault Prevention (CSAPs) to increase community involvement and commitment to social change, thereby preventing sexual assault. In support of these efforts, ORS provides evaluation technical assistance and training to local CSAPs, including webinars. These web-based trainings are on substantive evaluation topics including evaluation plan development, online survey best practices and qualitative data analysis. Each training includes practical exercises and informative handouts.