Friday, December 2, 2011

You can't manage what you can't measure. Not at scale, anyway.

A year ago I wrote, re investing in social marketing, "The social loop will share superficial characteristics with the display loop, but it's really completely different... the area with the most near-term leverage will be tools that help communicators understand the impact of how they are communicating and then help them make better decisions." This has turned out to be completely true.

I've been thinking about social marketing for five years. It has seemed obvious that major advances in marketing technique will occur through the social channel, but it was never clear to me exactly what those would be. I looked at and worked with a couple dozen social media marketing companies before throwing up my hands and declaring non-prescience.

My rule of thumb is that when the evolution of the landscape seems unknowable it is usually because the technology that will underpin the advance is still in flux. The obvious solution is dropping a level deeper in the stack and looking for investments there. In mobile, that meant Flurry four years ago and Media Armor a year ago. In social, it meant Awe.sm.

The smartest guy I ever knew in the ad business (like being the tallest dwarf, I know...) said, of managing people, "Whatever chart you put on the wall goes up."
That was me, the tallest dwarf, from back when I knew Clay, when he was just another guy.

I worked at IBM during the heyday of the Six Sigma movement. I was a design engineer, trying to optimize a very small piece of the central processor of what became the System 390 series of mainframes. As a design engineer there were several layers of abstraction between me and the silicon: the design language was a visual one--I wrote a flowchart which was compiled into a set of logic gates which were then mapped onto silicon. Aside from tweaking the logic gate-level design to try to get better performance, I spent my time at the flowchart level, as did most of the engineers.

Six Sigma methodology has you measure processes, find causes of errors and remedy them. The idea is to improve processes until there are fewer than 3.4 defects per million. IBM had a company-wide mandate to implement Six Sigma. I was subject to this mandate.

I asked my manager how I was supposed to measure my 'defects' and why would I even want to if I had to define them in such a way that I essentially never, ever made that type of mistake. He said "How are you going to improve if you aren't noticing your mistakes and figuring out how to stop making them?" "I already do that," I said, "I'm just not marking them down on some stupid piece of graph paper thats been pre-printed with a normal curve." He said "But then how can we manage it?"

Ah, Bach.

You can't manage what you can't measure. Stupid as managing designers on the binary idea of defect/not-defect and on such a stringent scale, constantly knowing how well you are doing so that you can constantly improve is extremely powerful. This idea, probably more than any other, drives my investment strategy: things that are not being measured are being managed poorly; creating new ways to measure creates ways of doing things immensely better, it creates entirely new businesses.

The fact is, you do get what you measure, whatever graph you put on the wall will go up. But the moral of that pithy aphorism was meant to be: be careful what you wish for.

If what you are measuring in social marketing is Likes or Follows, that is what you will get. But how closely aligned are these measures with what a business really wants: happy and loyal customers, higher sales? You don't know. No one knows. This particular loop hasn't been closed. Because the social gesture cause and business result can't be tied together in a measurable way, it can't be managed and it can't be improved.

I invested in Awe.sm's seed round because they provide core social measurement functionality, the ability to tie social actions into their actual results, to close the loop. I re-upped into their Series A because they're now doing something even more interesting: they're providing this functionality to other developers via API. Instead of being just an analytics player, they're now enabling the creation of an entire social marketing infrastructure that can use measurement to provide a ever-improving feedback loop.

I may have gravitated to marketing in part because dealing directly with people is too messy to ever even approach Six Sigma, but the engineer in me still believes that by measuring you can improve, and by linking measurement and algorithms you can create a feedback loop that allows you to improve adaptively and in real-time. This idea has revolutionized online advertising over the past few years. It's going to revolutionize social marketing over the next few.