Predict the Social Impact of Your Program

October 29, 2014

mismeasAs funders call for a new set of questions regarding the overall value of the programs they fund, nonprofits everywhere are challenged with a new reality, a reality that requires new evaluation tools and techniques to advance the social impact of programs. The Impact Genome Project (IGP) arrives at a time when funders are asking whether a program advanced the appropriate outcomes, and how the program compares to others.

In his recent webinar, Using Big Data to Predict Social Impact, Jason Saul, one of the nation’s leading experts on measuring social impact, challenges the social sector to raise the bar and come up with a new generation of data that allows us to answer a fundamentally different set of questions. This will require changing the way we approach evaluation, a shift from the overall obsession with measurement and accountability (output metrics) to a focus on what actually creates value and achieves the desired result that brings outcomes to the forefront (efficacy indicators).

Saul’s organization, Mission Measurement, helps corporations, nonprofits and the public sector measure and improve social investment. Mission Measurement launched IGP this year to shift the results of evaluation away from the retrospective model to a predictive model. Saul notes that the social sector is the last sector not to use predictive data to make better decisions, even though we are likely the sector with the richest data.

The challenges we face in the social sector include:

  1. lack of a common language
  2. measuring every single output or performance metric of our program (instead of measuring what determines our contribution to outcomes and how far we are moving the needle on those outcomes)
  3. the focus on measurement as a primarily academic inquiry – evaluation we can’t understand sits in shelves, websites and databases

Saul urges the social sector to build on the great evaluation work we have been doing and codify that knowledge to improve the level of practicing in the sector.

With all the academic knowledge that exists and practical data of what works, what doesn’t, and what drives outcomes, we can dramatically lower the cost of evaluation. Saul calls for organizations in the social sector to share knowledge among funders, nonprofits, academics, government officials and corporations.

Sharing knowledge and creating data and benchmarks that are available to help people make better decisions as a sector can diminish the burden and costs of evaluation. And instead of hiring pricy evaluators for every single program or in some cases letting evaluation happen to us, we can produce much better outcomes and allocate resources much more efficiently to organizations that are producing results or effective interventions.

To learn more about the Impact Genome Project (or propose a genome project that matters to you), Jason Saul, and the work of Mission Measurement, visit its website.

– Naomi Marx, MCF executive assistant

Fast Forward: Phil Buchanan on Effective Practice in Philanthropy

May 27, 2014

Screen Shot 2014-04-23 at 12.09.07 PMThe newest edition of MCF’s Fast Forward series is out!

In this episode, our president Trista Harris speaks with Phil Buchanan, president of The Center for Effective Philanthropy (CEP). CEP is a nonprofit that helps foundations assess and improve their effectiveness and performance. Phil identifies the four components to effective practice as:

  • Clear goals
  • Coherent strategies
  • Disciplined implementation
  • Relevant performance indicators

Phil and Trista discuss the challenges foundations face in receiving honest feedback, tools CEP has developed to address those challenges, and his advice to foundations interested in becoming more effective organizations:

Read the transcript of the interview on our website, then stop by the Fast Forward main page to catch last month’s interview with Margaret A. Cargill Foundation’s Mark Lindberg if you missed it!



The Path to Effective Philanthropy: Honest Conversations

February 27, 2014

It was a treat this week for MCF and our members to host a conversation with Phil Buchanan, president of the The Center for Effective Philanthropy (CEP).

Our exchange was energetic, provocative, fun and sometimes funny.  I was struck by how often we circled back to core concepts and philanthropic fundamentals.

The discussion aligned serendipitously with MCF’s Principles for Grantmakers.  Here are a few snippets that illustrate the challenges — and opportunities — of putting principles into practice.

Effective Governance
According to MCF’s Effective Governance Principle, grantmakers are expected to be good stewards of assets, to fulfill donor intent, to make sound decisions and to perform all fiduciary responsibilities.

Buchanan called for foundation boards to govern effectively by not rubber stamping staff members’ grant recommendations.  “If the board is approving every grant, they’re not taking time to see what it all adds up to and they’re not asking the hard questions.”

And he challenged foundation CEOs to practice “radical openness” with their boards – i.e., to say everything they’re thinking and to spark “messy conversations.”  Good governance doesn’t emerge from perfectly scripted board meetings at which “the most spontaneous thing that happens is when someone gets up to get a cup of coffee.”  (Yes, it’s okay to laugh at ourselves.)

Engaged Learning
The MCF Engaged Learning Principle calls for continuous learning and reflection by engaging board members, staff, grantees and donors in thoughtful dialogue and education.

Of course, learning and continuous improvement through performance assessment is at the heart of CEP’s mission.  (Buchanan readily acknowledged that he is not the expert in philanthropy . . . and he cautioned us to be wary of those who say they are.)

Because philanthropy is “wicked tough,” funding programs on theory alone is not enough.  It’s vital that grantmakers establish performance indicators and are data driven.

And they sometimes need to follow, not lead.  By replicating proven programs, foundations can learn from others and succeed.  (For more on shared goals, read Buchanan’s opinion piece in this week’s Chronicle of Philanthropy.)

Through MCF’s Transparency Principle grantmakers strive to build healthy relationships with the public, applicants, grantees and donors by using clear, consistent and timely communications.

Being transparent includes sharing the so-called “failures.” (Our host Kate Wolford of The McKnight Foundation noted that we might be more apt to learn from our missteps by reframing them in more positive, multi-dimensional terms.)

Buchanan reported that it’s up to foundations to share the results of CEP assessments.  Some don’t share at all, some partially share with grantees (and sometimes add a positive spin!), and some share widely, warts and all.

He noted that foundations that are truly transparent are viewed as trustworthy and credible.  For example, Robert Wood Johnson Foundation is fully committed to evaluation and transparency, making it all the stronger.

Cynics may say that foundations don’t need to be accountable to anybody.  But as Buchanan reminded everyone, if grantmakers aren’t honest and don’t cultivate positive relationships with their grantees, how can they obtain the candid information they need to improve philanthropy . . . and improve lives?

More to Come
Keep watching our Philanthropy Potluck Blog for future postings about philanthropic effectiveness, including video conversations with MCF President Trista Harris, Buchanan and other big thinkers.

Like our grantmaker members, MCF is committed to hosting robust conversations within and across sectors . . . because leadership for the 21st century requires honest, provocative discussion.

— Wendy Wehr, MCF vice president of communications and information services

Minnesota: A Case Study In Large-Scale Data Collection and Impact

January 29, 2014

GiM_mediumThe blog Markets for Good, an effort by the Bill & Melinda Gates Foundation and others to improve the generating, sharing, and acting on data in the social sector, just interviewed MCF President Trista Harris about our annual Giving in Minnesota report. We are reproducing the entire Q&A here; you can also read it on the Markets for Good website. Thanks to them for the opportunity!

What data collection techniques and technology does MCF use to produce its annual Giving in Minnesota report?

Producing Giving in Minnesota is a year-round process. Production of the 2013 Edition began in 2012, when Research Associate Anne Graham began identifying the top 100 grantmakers in the state and asking them to provide their grant lists for coding.

Our process succeeds because of the human element. Over the past eight years Anne has built strong relationships with Minnesota’s top grantmakers – including corporate grantmakers who are not required to file tax forms that make public specific details about their annual giving.

And we accommodate the grantmakers. They can submit their grants lists in whatever form is easiest for them – whether it’s an electronic file pulled from their grants management system or a hard copy of their annual grants list.

Of course, even with this high-touch, low-tech approach, we do need to rely on some public sources to complete our data collection. But here again, we have a close working relationship with individuals in the office of the Minnesota Attorney General, and they give us direct access to a database that allows us to update grantmakers’ financial data and identify new foundations in the state.

And our technology? Microsoft Access relational databases still serve us well for data management, annual reporting and long-term trend analysis.

What recommendations would MCF have for other organizations seeking to collect critical information directly from many sources?

MCF may be the only U.S. regional association of grantmakers that is collecting grantmaker data on this scale. To maintain our organizational commitment to this research work (since 1976), we have fully integrated it as a core member service.

Producing Giving in Minnesota goes hand in hand with our other research projects. We communicate closely with foundations and corporate grantmakers to publish our Minnesota Annual Grantmaker Rankings. Plus, we connect regularly with these same organizations to keep our Minnesota Grantmakers Online (MGO) database of grantmakers and grants current for nonprofit grantseekers.

And our lead data coder knows her stuff, too. Having worked for MCF for more than 10 years and in the state’s nonprofit sector longer than that, she knows minute details about Minnesota’s grantmaking and grantseeking scene.

While it would be hard to achieve that level of coding consistency and reliability from someone outside Minnesota, we are sticklers for maintaining national-level standards that enable comparisons between our data and others’ around the country. In coding, we use the National Taxonomy of Exempt Entities (NTEE) system and the Foundation Center’s Grants Classification System (GCS).

What are the benefits of this research for grantmakers and others in the community? What are the limitations?

MCF members rank our research as one of the most valuable benefits of membership. Using this source of reliable data, MCF staff can credibly explain the value and scope of philanthropy in our state. We publicize our findings to nonprofit grantseekers, business leaders and representatives of the media. Plus, we overlay US. Congressional district information so we can use our data to reach government officials and support our public policy work.

Having a robust database and staff expertise in-house, MCF can quickly respond to custom requests from grantmakers. For example, Anne goes deep into the data to help members benchmark their giving against peers, understand geographic shifts in funding, and sift through dozens of subject area subcategories to answer cross-subject and cross-population funding questions – such as which pockets of human services and health giving benefit seniors.

Having local data, we were able to report on how grantmakers responded to the Great Recession by shifting dollars from education to human services. We can see what grants are benefitting the metro area around Minneapolis and St. Paul and what are directed toward rural areas.

And we’ve been able to document the full value of corporate giving in Minnesota (which has the second highest number of Fortune 500 companies per capita in the country). Our state’s corporate grantmakers, including multi-nationals such as Target and General Mills, account for less than 10 percent of grantmakers but give 43 percent of the annual grant dollars. But they also direct more giving beyond our state’s borders than do our private or community foundations.

Still, what we know from our research is sometimes dwarfed by what we don’t know. Just as others across the country struggle with collecting beneficiary data, so do we. Receiving incomplete grant descriptions, we significantly under-report support for specific population groups.

And we receive complaints about the lag time in reporting complete information (which is also something that plagues the sector as a whole). Our grantmaker members often ask for real-time information so they can learn about the strategic intent of their peers and identify collaborators. At the end of the day, though, we cannot collect and analyze data that the grantmakers themselves don’t track.

But let’s not lament the gaps in our data. Instead, let’s use our rich data repository to identify trends that enable us to see into the future and succeed as 21st century philanthropic leaders. In my recent Philanthropy Potluck Blog post about predictions for the future, I envision how MCF’s data, our members’ data from grantees, and data from other sources such as Minnesota Compass can be combined to create a positive force for change in our state.

Evaluation Capacity That Sticks

May 29, 2013

stateofeval1Don’t miss MCF’s spring issue of Giving Forum, online and in your mailbox now, to see how consulting firm TCC Group recommends that grantmakers work with nonprofit organizations to build evaluation capacity that sticks.

According to Innovation Network’s State of Evaluation 2012 report:

  • 90% of nonprofits evaluate their work,
  • but only 28% have promising capacities and behaviors in place to meaningfully engage in evaluation.

The barriers to nonprofit evaluation capacity include the usual suspects: limited staff time, insufficient financial resources and limited staff expertise.

PeiYao Chen, associate director of evaluation at TCC Group, offers three strategies for funders that want to play a role in strengthening nonprofit evaluation capacity, and she recommends that nonprofits work together and use more affordable cohort consulting, to get it done.

To learn much more, read “Building Evaluation Capacity That Sticks” in Giving Forum.

– Susan Stehling, MCF communications associate

Evaluation: Make It Meaningful and Useful

May 17, 2013
Michael Quinn Patton

Michael Quinn Patton

Don’t miss MCF’s spring issue of Giving Forum, online and in your mailbox now, for “Making Evaluation Meaningful and Useful” by Michael Quinn Patton.

He says, “High-performing organizations make evaluative thinking a way of doing business.”

He distills 40 years of experience conducting evaluation, training evaluators and writing about evaluation into three important lessons. Here’s a taste. There’s much more in the complete Commentary.

  1. Embedded Evaluative Thinking Creates Lasting Impact: Patton stresses that the first step is distinguishing evaluative thinking from doing an evaluation and says, “Evaluation is an activity that produces reports; evaluative thinking produces effective organizations.”
  2. Evaluation is a Leadership Responsibility and Function: Evaluation must not be seen as a technical or administrative function. Instead it is an ongoing inquiry into what works — for whom, in what ways and under what conditions– that must become a strategic priority.
  3. Evaluating Your Organization’s Evaluation Culture Deepens It: This step involves asking tough questions of your staff: How is evaluation viewed here? How are failures handled? What would you tell a new co-worker about how to approach evaluation?

He also shares a bit about his work over the last year with the Otto Bremer Foundation, an MCF member, as it works to embed evaluative thinking into its culture.

He mentions a set of 25 Evaluation Flash Cards that the foundation put together to summarize key evaluation concepts and their implications for grantmaking. The flash cards will soon be available on the Otto Bremer Foundation website as a resource for the philanthropic and nonprofit community. We’re anxious to see them too, so we’ll let you know when we hear that they’ve been posted.

The spring issue of Giving Forum is all evaluation, measurement and results, so don’t miss it!

– Susan Stehling, MCF communications associate

Connecting Investment With Impact

May 14, 2013

GTCUWlogoDon’t miss MCF’s spring issue of Giving Forum, online now and in your mailbox, for a look at how Greater Twin Cities United Way‘s community investment strategy has evolved over the years in “Connecting Investment With Impact.”

In the article by Brian Paulson, director of innovation strategies at United Way, you’ll learn how the organization has gone from:

  • measuring activities
  • to focusing on outcome measures and building evidence
  • to creating emerging models of systems integration through collective impact.

And, you’ll be privy to lessons learned along the way.

For the complete piece, don’t miss the spring issue of Giving Forum.

– Susan Stehling, MCF communications associate