[Editor's note: Also read about Starbucks' newly released sustainability report here.]
In recent years, we've seen the explosion of supply chain management programs and eco-labels assuring buyers that products are sustainably produced. The Ecolabel Index currently tracks 424 different types of ecolabels in 246 countries and 25 industry sectors -- 145 related to food alone.
While many companies have also started developing their own supplier certification programs tailored to their specific needs and operating environment, few programs (even ecolabels) have devoted resources to ongoing monitoring of the programs.
With so much financial investment in these programs, many have begun to ask: Have the programs been effective? Are they driving sustainable production in the sector and providing some of the social benefits they promised? Recently, Conservation International (CI) collaborated with our longtime partner, Starbucks Coffee Company, to begin answering these questions for its home-grown supply chain program, Coffee and Farmer Equity (C.A.F.E.) Practices, and we have a few takeaways to share on the value of these assessments as well as how to build ongoing results assessment into your own program.
Launched in 2004, the C.A.F.E. Practices program includes a set of social, environmental, economic and quality objectives and related guidelines for the production and processing of coffee that are assessed in the field by a third party. CI has advised Starbucks on the program since its inception, and together, we developed a continuous improvement model designed to keep producers in the program and increase their scores on an ongoing basis over time. In 2011, Starbucks purchased 367 million pounds of coffee -- or 86 percent of their global purchase -- through its C.A.F.E. Practices program. (You can find CI's logo on the package of verified coffees in any Starbucks store.)
In 2009, CI began to produce annual global reports, which are assessments of the data provided in verification reports from all participating countries. CI also conducted field surveys with farmers participating in C.A.F.E. Practices and farmers not participating in the program in two sample regions, Guatemala and Colombia, to supplement and validate the findings of the global reports.
Our most recent global report covering 2009-2010 was released this month and the full report and summary of program results can be found at Conservation International's website.
Through our experience with Starbucks, we have found that these types of assessments provide many benefits to the company, including specific management-level information that helps them improve the supply chain verification program and its impacts on the ground through its Farmer Support Centers.
The results assessment also gives Starbucks confidence that their investment in the program has been worthwhile. (Read my CI blog to learn more about the assessment findings and the ways Starbucks is using this information to improve C.A.F.E. Practices.)
Best practices in results assessment are emerging as more stakeholders are expressing interest, but we've found a focus on a few key design elements is critical:
- Plan for assessment at the outset. Incorporating a monitoring and assessment framework into the design of the program itself helps to ensure the necessary data collection and management procedures are in place and are aligned with existing procedures -- a strategy that helps to reduce costs.
- Simplify. There are myriad indicators we might wish to evaluate as part of our assessments, but assessment can be costly -- especially field-based assessment. Focusing on the most critical indicators that really provide the information you're looking for and eliminate duplicate questions helps to keep costs contained.
- Provide room for open-ended investigation. Some program benefits are not easily quantified. Designing assessments with this in mind and providing participants room to express opinions about topics outside of the survey questions will help you capture these types of benefits and challenges.
Although much of the interest in sustainably produced goods is currently focused on the food and agriculture sector, the importance of assessment is relevant across sectors.
It is no longer enough to measure success by whether production or purchasing according to a standard or supply chain management program is growing. Instead, we should turn our focus to whether the standard is effective in delivering the results it was designed to achieve. With that assurance and with the nuance the assessment provides, we can ensure that every dollar invested in these programs makes a difference.
To see how Starbucks' C.A.F.E. Practices have impacted coffee farmers in Chiapas, Mexico check out Team Earth Chiapas, Team Earth is a series of videos and stories about the results Conservation International has been able to achieve through partnerships with other nonprofits, governments, and businesses.
Starbucks cup photo CC-licensed by Piero Fissore.