What international development can teach businesses about metrics

Questioning Assumptions

What international development can teach businesses about metrics

Image courtesy of United Nations.

[Editor's note: This is the third in a multi-part series that examines the pitfalls of sustainability measurements while drawing on lessons learned from outside the business world. For additional context, see the first installments here and here.]

Since this series' last installment, where we identified several measurement pitfalls the sustainability world should avoid, GreenBiz released the 2013 State of Green Business report and a corresponding new set of measures to capture it.

The report describes how businesses are transforming for the new normal of a world that is increasingly volatile, climatically unstable and interconnected.  

So as a reflection of our changing world -- and sustainable business along with it -- we chose one complex problem for Part III's "Lessons Learned" about metrics: global poverty.

Since 2000, the International Development (ID) community's battle to end (or substantially reduce) poverty by 2015 has played out on the world stage. There have been delays, defeats and some solid accomplishments. In doing so, the development community's thinking has evolved about what best constitutes effective aid and how to know whether it's working.

We find significant lessons for the sustainable business world in the story of the Millennium Development Goals (MDGs) and the groundbreaking shift towards metrics that occurred with the 2008 Accra Agenda for Action. The ID community has been considering questions of direction towards greater levels of sustainability and how to measure their actual effectiveness over 13 long years of debate and trial and error. We could learn from their longer experience with sustainability goals and metrics.

Pitfall 5: Setting and starting out on goals without doing the groundwork to assess what -- and who -- are being left out. As sustainability can be more complex than it initially appears, goal setting while pursuing it can be even more challenging.

The ID field experienced painful growing pains from what happens when goals, and the initial metrics chosen to evaluate progress towards them, fail to capture what's really important for all stakeholders.

In 2000, the United Nations set eight very ambitious goals to address poverty, one of the world's most intractable problems, by 2015. The MDGs divided the challenge into goals focused on achieving education, promoting women's rights, improving child and maternal health, combating diseases and ensuring environmental sustainability.  

From the beginning, the MDGs drew criticism, starting with the justifications for each of the eight chosen goals. The corollary targets and dates were criticized for lacking rigor.

The MDGs goals did not account for broader issues, such as developing countries' capacity for governance, state of infrastructure and structural causes of poverty. And they did not include clear guidance on what measurements would be used to mark progress.

In the years since the MDGs were developed, the international community has met several times to discuss, argue and agree on remedies to accelerate what has been uneven progress towards the goals. The 2008 United Nations High Level Forum on Aid Effectiveness produced the Accra Agenda for Action, a paradigm-shifting outcome.

This agenda set new commitments to transparency, accountability and demonstrating the results of aid. It can be argued that Accra was the first time the development community agreed to a metrics or evaluation-based approach. You can see this now, among other places, in the emphasis USAID puts on measurement.

In "More than Good Intentions: How a New Economics Is Helping to Solve Global Poverty," Karlan and Appel took this further: "Until recently we knew astonishingly little about what works and what doesn't in the fight against poverty." Their work explores in depth this now well known and still highly debated problem.  

Interestingly, while this strongly resembles businesspeople wanting to know if their sustainability efforts are working, the development community seems to be taking it much deeper. They are asking sophisticated questions: "Maybe the numbers are better now, but how do we really know whether it was our efforts which led to this improvement? Perhaps it was due to other factors which we've overlooked."  

The only way to know would be to do a careful experiment with one group subject to a strategy anticipated to improve the situation, and a matched control group which is not.  

Businesspeople, in our experience, don't tend to ask this question, and we're not saying they should -- yet. But, if so, then there are even more potential lessons to be learned from this field.

Readers: Do you see this need to more solidly understand causal connections in your sustainability efforts between goals, actions and results, as opposed to being limited to what might be, in effect, correlational assessments?

Other lessons, though, are more certain.

Would it have been preferable for these steps to have been hammered out at the beginning? Definitely. Were these outcomes eight years late? Perhaps. But, most important, the development world moved forward. This work proved robust enough to be a laboratory to discuss, argue and critique what had been done in order to get better.

Bringing things up to date, as the 2015 deadline looms, the U.N. is working to build on the MDGs so that the next iteration is better and more inclusive. These will be known as the Sustainable Development Goals. Their process explicitly addresses "the shortcomings of the MDGs," including the ones above.   

Further, and very important, the new goals being discussed will focus explicitly this time on sustainability. In contrast, the MDGs had only one of its eight goals focused on environmental sustainability.  

Pitfall 6: Numbers are not necessarily unambiguous in what they say

A deep lesson of a complicated endeavor such as the MDGs is that the numbers don't necessarily tell you if you failed. For example, champions and critics of an aid initiative such as the Millennium Development Villages (MDVs) come to very different conclusions -- even when using the same data. See some of the reasons here, here and here.

At the end of the day, the ID field is still arguing about whether the MDVs worked, and these dualists are statistically trained Ph.D. economists like Sachs and Easterly.

Now, as we also recognize and get further into the complexities of sustainability, we also might find our metrics do not unambiguously tell us if our company's sustainability efforts are fundamentally succeeding. But that doesn't indicate a wasted effort.

They might still help us better understand, albeit imperfectly, how we're doing and where we need to improve. It is important, though, not to make the assumption they will eliminate the hard work of interpretation, and to try to recognize and minimize our biases. A spirit of inquiry, a bit of humility and not setting unrealistic expectations for what our metrics will do for us, before the bean counting starts, also would be helpful.  

Pitfall 7: Missing the value of the metrics process

Just as the MDGs are evolving to become Sustainable Development goals, encompassing a deeper and more comprehensive sense of sustainability, we too should seek to frame and express the company's goals as statements of what's important. As the ID experience has shown, the discussion itself has value (which is not to say it has to take so long, or be quite as contentious). But it is important to surface and question any prevailing assumptions that: (a) Sustainability is fundamentally just "business-as-usual" for our company; (b) "success" at the end of the day is just what it's always been; and (c) now, we're just being asked to simply measure it. The process also should encourage discussions about the question:  "Are we missing something really important?"

After that, we could move to how to go about accomplishing the goals.

Only then can we consider how a possibly rethought conception of success should be measured. Otherwise, we risk treating metrics as little more than a numbers and charting exercise, or having it come across that way. A consequence of primarily focusing on the math is that we won't succeed in engaging enough of the mind and heart muscles hidden in our company, some of whose owners might like to be invited in if it's not seen as just for "data people."  

As the Sustainable Business Metrics field matures, we have the opportunity take advantage of the available lessons, not just from the ID field, but from the patterns emerging from a number of fields as we go further into this series. We don't have to risk making the same mistakes.

The next parts of this series will offer more examples from international development, psychology, political science, systems thinking, sports and even weather forecasting.

Image courtesy of United Nations.

Topics: