How eBay's radical efficiency plan benefits from employee engagement
To hear more about how Dean Nelson is engaging eBay on data center efficiency and power sourcing, don't miss VERGE SF Oct. 14-17.
This week, eBay will take the wraps off a data center in Utah that is the first facility of its kind to use fuel cells from Bloom Energy as its primary source of onsite power.
It's just the latest demonstration of the e-commerce giant's radical approach to data center design. That master plan aims to create the highest possible productivity for its roughly 120 million customers -- who generated $175 billion in eBay-enabled commerce during 2012 -- while having the lowest possible impact on the planet.
"We want to make sure that our actual carbon emissions that are coming out of each one of these things is as low as possible," says former Sun Microsystems engineer and real estate guru Dean Nelson, who officially is eBay's vice president of global foundation services. (Not a hint of sustainability in that title.) "Then, that rolls back more into the renewable energy pieces of it."
Over the past four years, Nelson has inspired eBay to push the envelope in both energy efficiency and clean power sourcing. Its "Project Mercury" modular data center facility in Phoenix boasts one of the best power usage effectiveness (PUE) measures around, an average of 1.2 depending on the time of year, despite being in the desert. (PUE measures the ratio of electricity needed for cooling IT equipment versus the amount of power needed to run the servers, network and other infrastructure gear. The "ideal" score is 1.) Despite its hot location, the facility can use outside air to "free cool" the technology year-round.
"Project Topaz" in Utah boasts similar innovations, including innovative water reuse strategies as well as the company's biggest on-site solar installation at 665 kilowatts. The swatch of panels covers 72,000 square feet, contributing about 10 percent of the power needed to run the facility. eBay actually lobbied for legislative changes within the state in order to make the project happen.
Much of this was somewhat invisible to the average eBay developer or product manager until earlier this year, when Nelson unfurled a Digital Service Efficiency (DSE) dashboard that helps interpret the link between IT infrastructure efficiency and the company's performance along cost, revenue and environmental lines.
The methodology (PDF) is applied to pools of servers across the company (about 52,000) and that visibility already has resulted in dramatic results, Nelson says.
During the second quarter, eBay's carbon impact per 1,000 URLs was down by 2 percent, while the number of URLs visited per kilowatt-hour rose 12 percent. The DSE metric has enabled teams across the 30,000-person company to gain much deeper insight into the impact of their individual actions and decisions.
Nelson's team actually went so far as to create two internally published lists that pinpoint the most and least efficient servers across the company: the Hall of Fame highlights infrastructure worthy of emulation, while the Hall of Shame identifies servers that need to be either retuned for better efficiency or decommissioned.
"That drove tons of dialogue internally about utilization, about disaster recovery, about spare capacity, the age of servers, a whole bunch of dialogue that really drove program to drive overall system performance and consistency," he says.
How big of an impact can this have? During the first quarter, when one eBay developer made a simple adjustment in the amount of server hardware memory required by a piece of cloud software code, the company was able to shut off 400 servers. That may not seem like a lot, but it helped the company avoid $200 million in cost avoidance (because the servers didn't need to be refreshed or replaced), reduce the power draw required by the related data center by 1 megawatt and eliminate approximately $1 million in operational expenditures. Now, imagine multiplying that impact by the thousands of eBay developers writing software code for the platform.
"That visibility changes behavior," Nelson says. "If you can't see it, how do you tune for it? Now, we have a way to measure the entire fleet of equipment we have, how well it is deployed inside, and then how it is being used. Just like driving your car."
Nelson credits his ability to connect the dots for eBay's employees with his varied background at Sun, where he started out debugging chip level components on the company's manufacturing floor after earning a technical degree at DeVry University in Phoenix. Eventually, he worked up through the Sun ranks into his final role managing real estate and technical infrastructure for the multi-billion-dollar research and development organization. "That's where I actually started seeing the effects of the equipment that was purchased on results," he remembers.
Since releasing the DSE methodology, eBay has had discussions with many other companies across the high-tech ecosystem interested in applying the same concepts to their own business. Nelson long has been inclined to share his best practices: he was the founder of Data Center Pulse, which represents data center owners and operators, and he is also a technical advisor for the Green Grid. Meanwhile, he and his team already are envisioning ways in which DSE can be improved and extended, and they are working on key performance indicators that can be used globally.
"The more visibility we get internally, at every layer, the more knobs that come up that are available to turn," Nelson says. "When you start getting people excited because you have shown them the implications of their decisions, there are just amazing things that can happen. The conversations internally, this quarter, are so different than they were last were because people have context, they understand how the dots connect, and they can really start driving some extreme efficiency decisions and some extreme productivity decisions."
This story has been updated since it first published.