You can't manage what you can't measure. Not at scale, anyway.

A year ago I wrote, re investing in social marketing, "The social loop will share superficial characteristics with the display loop, but it's really completely different... the area with the most near-term leverage will be tools that help communicators understand the impact of how they are communicating and then help them make better decisions." This has turned out to be completely true.

I've been thinking about social marketing for five years. It has seemed obvious that major advances in marketing technique will occur through the social channel, but it was never clear to me exactly what those would be. I looked at and worked with a couple dozen social media marketing companies before throwing up my hands and declaring non-prescience.

My rule of thumb is that when the evolution of the landscape seems unknowable it is usually because the technology that will underpin the advance is still in flux. The obvious solution is dropping a level deeper in the stack and looking for investments there. In mobile, that meant Flurry four years ago and Media Armor a year ago. In social, it meant Awe.sm.

The smartest guy I ever knew in the ad business (like being the tallest dwarf, I know...) said, of managing people, "Whatever chart you put on the wall goes up."

-- Clay Shirky

That was me, the tallest dwarf, from back when I knew Clay, when he was just another guy.

I worked at IBM during the heyday of the Six Sigma movement. I was a design engineer, trying to optimize a very small piece of the central processor of what became the System 390 series of mainframes. As a design engineer there were several layers of abstraction between me and the silicon: the design language was a visual one--I wrote a flowchart which was compiled into a set of logic gates which were then mapped onto silicon. Aside from tweaking the logic gate-level design to try to get better performance, I spent my time at the flowchart level, as did most of the engineers.

Six Sigma methodology has you measure processes, find causes of errors and remedy them. The idea is to improve processes until there are fewer than 3.4 defects per million. IBM had a company-wide mandate to implement Six Sigma. I was subject to this mandate.

I asked my manager how I was supposed to measure my 'defects' and why would I even want to if I had to define them in such a way that I essentially never, ever made that type of mistake. He said "How are you going to improve if you aren't noticing your mistakes and figuring out how to stop making them?" "I already do that," I said, "I'm just not marking them down on some stupid piece of graph paper thats been pre-printed with a normal curve." He said "But then how can we manage it?"

Ah, Bach.

You can't manage what you can't measure. Stupid as managing designers on the binary idea of defect/not-defect and on such a stringent scale, constantly knowing how well you are doing so that you can constantly improve is extremely powerful. This idea, probably more than any other, drives my investment strategy: things that are not being measured are being managed poorly; creating new ways to measure creates ways of doing things immensely better, it creates entirely new businesses.

The fact is, you do get what you measure, whatever graph you put on the wall will go up. But the moral of that pithy aphorism was meant to be: be careful what you wish for.

If what you are measuring in social marketing is Likes or Follows, that is what you will get. But how closely aligned are these measures with what a business really wants: happy and loyal customers, higher sales? You don't know. No one knows. This particular loop hasn't been closed. Because the social gesture cause and business result can't be tied together in a measurable way, it can't be managed and it can't be improved.

I invested in Awe.sm's seed round because they provide core social measurement functionality, the ability to tie social actions into their actual results, to close the loop. I re-upped into their Series A because they're now doing something even more interesting: they're providing this functionality to other developers via API. Instead of being just an analytics player, they're now enabling the creation of an entire social marketing infrastructure that can use measurement to provide a ever-improving feedback loop.

I may have gravitated to marketing in part because dealing directly with people is too messy to ever even approach Six Sigma, but the engineer in me still believes that by measuring you can improve, and by linking measurement and algorithms you can create a feedback loop that allows you to improve adaptively and in real-time. This idea has revolutionized online advertising over the past few years. It's going to revolutionize social marketing over the next few.

iMapBox

I've always been the type who, when confronted with a one-hour task, will instead take two hours to automate it. Here's an example.

VCdelta is my bot that tracks additions to VC portfolio pages. It has its own twitter feed. Its twitter feed is about to surpass my twitter feed in number of followers. It seems my bot is more interesting than I am. I thought it would be interesting to graph the number of people who have followed me versus the number of people who have followed VCdelta over time. Twitter does not provide stats like that, but whenever I get a follow email from Twitter, I hit archive, not delete. So all I needed to do was count the follow emails by month.

Turns out Python doesn't have a very good library for using a mailbox as a data source. The Python email libraries assume you are planning on writing an email client. So I wrote an abstraction layer for the Python IMAP library. Code is here*.

Here's the code to count twitter followers:

import IMapBox 

me=IMapBox.IMapBox("imap.gmail.com",my_acct,my_pwd)
mymail=me["[Gmail]/All Mail"]

myfollows=mymail.frm("twitter").subject("following")

mydates=[myfollows[x]['date'] for x in myfollows]

The 'me=' and 'mymail=' open a connection to my email account and select a mailbox, in this case the All Mail mailbox. (The command 'me.list()' lists all the mailboxes for the account.)

The next line filters mymail so myfollows is only emails from Twitter that have 'following' in the subject line**. iMapBox is lazy--it doesn't fetch the emails itself until it has to--so this is pretty fast. myfollows acts like a dictionary, so you can len() it, ask for the keys()--these would be the message IDs--or the items(), iterate over it, or get items.

Each of the items in the dictionary is an email message. These also act like dictionaries, with keys like 'from','to','subject','date', and 'text'. The next line creates a list called mydates of the date each follow email was sent. It does this by iterating over each item in myfollows and pulling its date out. This is the slower part: when you set up an iterator, iMapBox gets all the headers***.

The part about counting follows per date I will leave as an exercise to the reader. Here's the graph of my follows and VCdelta's follows. I've been tweeting for some three years, VCdelta for six months.

On a sidenote, this is a logarithmic scale. The green line is my trend. This is odd, no? I mean, I'm not getting exponentially more popular, so this argues that a lot of follow behavior is algorithmic of some sort. I had expected more linear growth.  I also expect VCdelta to level out soon, as it reaches the limits of its natural audience.

Another example, email volume over time:

You can see where I started using my current email account full-time, in September 2006. And you can see when I started investing full-time, in mid-2009. And you can see why my email response time has slowed dramatically.

The code:

from datetime import date, timedelta
import IMapBox

me=IMapBox.IMapBox("imap.gmail.com",my_acct,my_pwd)
mymail=me["[Gmail]/All Mail"]

for yr in range(2006,2012):
 for mo in range(1,13):
  beg_month = date(yr,mo,1)
  end_month = date(yr+mo//12,mo%12+1,1)-timedelta(days=1)
  print mo,"/",yr,"\t",len(mymail.dates(beg_month,end_month))

This is an alternative way to count emails per month, filtering by date instead of collecting dates. The 'dates(x,y)' method filters the emails for only those that were received between date x and date y (inclusive.) This is faster because even the headers are never fetched.

Some other ways to use it:

c=mymail.frm('josh')+mymail.frm('matt')
d=mymail.frm('josh')-mymail.to('matt')
e=mymail.today()
f=-mymail.today()

The first is all messages from either Josh or Matt. The second is all messages from Josh that aren't also to Matt, the third is all today's messages, the fourth is all messages except today's.

 ----- 
 * I'm an electrical engineer, not a computer scientist. So I can build a waveguide to your specifications, but I'm not entirely sure that this code is all that good. Please, feel free to fork, suggest improvements, make improvements, tutor me on garbage collection or unit testing, whatever. 
 ** I like object chaining. I know it's not Pythonic, but I'm not sure why. It strikes me that since I don't really understand too deeply how Python garbage collects, that this may be creating extraneous intermediate objects. If you plan to use this is any sort of real code, you might want to figure that out. I did notice that if I object-chain the IMAP connection ('me' in this example), it gets dereferenced and gc'd, which invoked the very polite __del__ method, closing the connection. I'm not sure how to avoid that, so I just commented out the __del__ method, leaving a messy open connection to the server. 
*** My thinking is to only go do the time-consuming fetching of messages when needed: when an email message object is referenced or when an iterator is set up (on the assumption that when you set up an iterator, you plan to consume the whole set of messages.) This latter is because fetching 100 messages in a single fetch is far faster than 100 single message fetches. The default is to only fetch the headers, except when the text itself is explicitly asked for. This default can be changed by setting priority='both' or priority='text' when you call iMapBox to open a connection to the server. 

Disruptive innovation, buy vs. build, the most pernicious lie in business, and how to know if you're fooling yourself

If a man has good corn or wood, or boards, or pigs, to sell, or can make better chairs or knives, crucibles or church organs, than anybody else, you will find a broad hard-beaten road to his house, though it be in the woods. 

—Ralph Waldo Emerson, big fat liar

No matter what the dictionary says, you can't describe a company as disruptive without giving weight to Christensen's description of innovation. It's perhaps overly simplistic to divide innovation into two categories--disruptive and sustaining--but the strikingly different characteristics of companies pursuing these strategies makes the partition a natural one.

Sustaining innovation means finding ways to do things better. Lowering the cost of manufacturing a widget by 10%, making a widget 20% more durable while only spending 10% more, reorganizing a department so ten people can do the work of twelve, creating an integrated supply chain to deliver goods to your stores in smaller quantities and less time. That sort of thing. Sustaining innovation often results in products that exceed customer needs at a given price point. The proliferating options in Microsoft Office show a sustaining innovation cycle that has exceeded most of the market's need.

Disruptive innovation means creating a product or service that is radically cheaper but much less functional (and this needs to appeal to a customer set that was previously underserved, so disruptive innovation often creates entirely new markets) and then using sustaining innovation to improve it until it meets mainstream customer needs (but is still radically cheaper.)

Before Google, there was targeted advertising. Very targeted. Hog Farmers Digest (now National Hog Farmer) was aimed at hog farmers. If you were a hog farmer, you read it; if you weren't, you didn't. It was a pretty effective buy: not a lot of wasted impressions. But creating an entire magazine for a very specific market is a difficult business proposition. The fixed cost of putting a book together limits how small its audience can be and so how targeted its ads can be.

Google's disruptive innovation was being able to create content for next to nothing. They can create a page that addresses a market segment as small as a single person for nominal marginal cost. Even though the content was lower quality than that it was competing with--the lack of human writers and editors means that any specific page is much less useful than a well-written and thought-out page would be--it turned out it was good enough. And because advertisers could be so specific in their buy, they could spend much less money. This opened up an entirely new market: advertisers that don't have multi-million dollar budgets.

Existing publishers could not compete: they could not lower their cost per page to anywhere near Google's. If they tried, they would lose quality and the loss of quality would mean losing their existing customers. This is the beauty of disruptive innovation: it is almost impossible for incumbents to respond. Disruptive innovations are disruptive because business logic precludes old-line companies from shrinking their business to address the disruptors.

It's incredibly difficult and expensive to challenge incumbents with nothing but a better product. Sustaining innovations are easy to copy and well-managed incumbents are always on the lookout for challengers and willing to learn from them. But when a disruptor comes along, they are trapped.

*****

What kind of innovation are we peddling in adtech? Article after article calls our companies disruptive, but do we really fit the Christensen mold? A disruption scenario would look like this:

• the existing industry would supply a product of higher quality/functionality than the majority of potential customers actually needs and at a very high price;
• the disruptive companies would find a way to bring in a product of lower quality/functionality at a much lower price;
• customers that did not need and could not afford the old product would emerge as customers of the disruptive product, allowing the new companies the wherewithal to quickly mature their technology until it was competitive in the old product's market.

Does this sound like ad tech to you? It doesn't to me. The current ad-world is not supplying services at a higher quality than its customers need and there seems to be advertising inventory at every price point. If you can't supply advertising at a radically lower price point to customers who were previously underserved at a quality level that the incumbents are not interested in touching, you aren't really in a position to be disruptive. Almost all of adtech now is sustaining innovation: building a better mousetrap.

We clearly have a better solution than what existed, no argument. But the big lie of business, the pernicious fallacy that has deluded countless entrepreneurs, is that if you build a better mousetrap the world will beat a path to your door. It doesn't work that way.

*****

What is going on in adtech right now is clearly innovative. But because it's not disruptive in the Christensen sense, it means we're going to have to earn our money. We need to move fast to build scale.

There have been scores of M&A discussions in adtech this Summer and only a few have resulted in deals. One of the things I heard as an excuse over and over (from buyers, from sellers, from bankers, from founders, after a few drinks) is that the buyer said "we don't need to pay up for this, we could build it internally."

Build versus buy is an interesting discussion to have before you buy anything, especially something with the revenue multiple adtech VCs are looking for. Cold hard fact is, there's almost nothing out there in adtech that someone else couldn't build from scratch. The CTO would certainly tell the CEO that building would be cheaper than buying a company, and be right.

And yet, and yet. And yet the companies that are prowling for bargains still can't get advertising right. They clearly have a ton of tech talent in their core businesses, and the ability to hire more. They have the money to hire and manage and build adtech solutions. But they don't. Why not?

When I was at Omnicom, back in the 90s, investing in the early interactive agencies--clearly not disruptive businesses--the old-guard ad agencies that then made up the bulk of Omnicom's business talked big about building their own interactive units. But they never could. They also refused to pay the valuations the i-agencies commanded. They were on the sidelines while their clients hired hotshot young startups to build their websites, and some of the startups got pretty big in the process.

There were several reasons for this. Primarily, the old guard couldn't hire good people: no one who understood the web back then would go work for an agency whose primary business was making 30 second films for TV. Why would anyone who was any good go be a second-class citizen at a firm that was paying nothing but a salary and had no career path in interactive? Why wouldn't they go instead to Razorfish and get stock options and be a hero to their management everyday? They would, of course, and they did. And almost all the true stars of that era spent time in one of the independent agencies.

As then as now. Why would any competent adtech engineer go work for AOL or Yahoo or Twitter or any of the other big old companies where stock options issued today will in all probability never be worth anything? There are plenty of good jobs at exciting startups where there's the possibility of making actual money*. More importantly, why go to one of those big companies and be a second-class citizen, the "ad guy," when at a startup you're essential to their product?**

Companies can do very well at their core mission. But when their core mission is media or software or infrastructure or professional services, it's going to be really hard for them to get a foothold in the quickly changing adtech world. This never seems to be taken into account in build versus buy analyses: they can't build, and even if they could, they won't. And if they do, it will suck. Trust me, I've been there. And if you don't trust me, just take a look around.

But remember that the era of the independent i-agencies only lasted some six or seven years. At some point the number of people that could do the work more than competently was enough that even old-line agencies could hire them. At that point the i-agencies were like every other agency: they competed head-to-head with the old guard. Many of the biggest remained independent until acquired for great prices. But these were the ones who earned it. Unlike a disruptive business where nothing but guts, an innovative spirit and a huge dose of luck are necessary, competing head-to-head means competing: blood, sweat and tears.

We need to keep building, ignore the distractions and focus on winning clients, not just raising money, so that when it comes time to compete head-to-head, we will win. That's as it should be, of course, and I think many of our industry leaders have what it takes. But if you're starting an adtech company and you want to win, you have to know that you're in it for the long-term. It's a marathon, not a sprint, the cliche goes, and it's true.

*****

Meh, you say. I'm disruptive, I am going to go viral, achieve imminent world domination and sell to Google for $5 billion in two years. Neumann's an idiot.

Maybe. But disruptive businesses have certain characteristics. Ask yourself these questions.

1. Am I creating a new market, bringing in a set of customers for whom there was previously no value proposition?

Disruptive businesses bring out a product or service that is so far off the industry price/quality line that customers who would never have used the industry's products start to. This gives the disruptor the foothold it needs to start improving quality until it threatens the incumbents. Google AdWords is an excellent example of this.

Who are the unserved markets in advertising? Are there any? I think there are, and I think that if you don't see any, you need to think about what advertising is more broadly.

2. What is price in my market?

If you're in ad-tech, what does price even mean to your end-customers (the advertisers***)? Is it just lower CPMs? There have always been low CPMs out there. Is it higher ROI? That's probably closer to the mark. The best answer I have heard is that it is lower risk: the ability to more accurately predict ROI.

You have to credibly answer this question and then be radically better along this dimension if you are disruptive. I think there are many answers here, and your answer will depend on your answer to question one, above.

3. What is quality in my market?

In disk drives (Christensen's first case study), this is an easy question: quality is how much data can be stored. The disruptors built lower-quality disk drives at lower prices, then used the march of progress to threaten the old-line disk makers. The old-line disk makers' customers wanted more storage, not less, so they did not see this market and could not address it with the existing customer bases. But key to the disruptors long-term value was the ability to improve quality quickly. If they could not, they would not have been able to displace the old guard.

What is quality in adtech? Conversion? Click-through? Pinpoint targeting? And if you know what quality is to your market, can you then improve quickly along that metric so you serve not only the new market you've created, but the giant market that already exists?

Quality. I've been thinking about this question for ten years and don't have a definitive answer. Do you?

If you do, if you think you really have a disruptive business model, call me, I'm looking to back people like you.

-----
* If this is you, email me.
** Soldiers don't get promoted if they haven't seen battle. If you want a career path, always take the job in the middle of the action, even if it pays worse.
*** And are the advertisers really your customers? Why aren't the 'consumers'?

To my friends who died ten years ago, I hope you had a fortunate rebirth.

To my friends who had family members die, may your loved ones find happiness.

To my son, born 42 weeks and one day later: there is only loss if there is love, the way forward is always through love.

How I Wrote VCBar

All the people ask me
How I wrote elastic man.
                     - The Fall

My friend Chris Wiggins asked me to post the code for the VC bar chart generator I blogged earlier this week. It's here.

It's an interesting project if only because it's run entirely on the client-side. There's no server side (except, of course, for delivering the files to you.) This is possible because the Crunchbase API supports JSON callbacks. Every bit of code in the git repo is as you see it on http://neuvc.com/labs/vcbar.

But in the spirit of making the source code available, I'm going to go one better and show you how to write your own visualization of Crunchbase data. Because there's no server-side, you can play with this code on your computer with nothing more than a text editor and a web browser.

Adapt this code to visualize other data sets: people respond to visualizations and, as this shows, it's not very hard to make them.

This code is going to be as bare as possible, no bells and whistles. I hope to illustrate just the bones of it. You can add bells and whistles and DTD declarations to your hearts' delight, but this works too.

*****

The program will be broken into three parts: the HTML, the CSS and the Javascript.

The HTML

Create a directory on your computer, download d3.js from https://github.com/mbostock/d3/archives/master, unzip the archive and move the file d3.js into your new directory. Then create a file named index.html in the directory. Put this in it:

<html> 
   <head>  
      <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.2/jquery.min.js"></script> 
      <script src="d3.js"></script>  
      <script src="vcbar.js"></script> 
      <link rel="stylesheet" type="text/css" href="vcbar.css" /> 
   </head> 

   <body> 
      <div id="barchart"> 
      </div> 
      <div id="controls"> 
         <a href="#" id="union-square-ventures">Union Square Ventures</a>
         <a href="#" id="true-ventures">True Ventures</a>
       </div> 
   </body> 
</html>

Pretty simple. The head loads the javascript (including jQuery from Google's CDN) and the CSS. The body has two divs, one named "barchart"--this is where the javascript will put the chart object itself--and one named "controls", where links for the two VC firms this example will link to will live. Note that the links do not link anywhere. We will use the javascript to execute an action when a link is clicked.

The Javascript, part 1: Getting and Parsing the data

Put all the javascript into a file called vcbar.js in the same directory as index.html.

There are three things we want to do in the program:
1. Detect when one of the links is clicked;
2. Get and parse the data;
3. Display the bar chart.

The first is easy, especially using jQuery:

$(document).ready(function () {
   $("a").click(function () {
      var vc = $(this).attr("id");
      $.getJSON("http://api.crunchbase.com/v/1/financial-organization/" +
               vc + ".js?callback=?",parseCB);
      return;
   });
});

This code uses jQuery (the '$') to run an anonymous function every time an <a> tag is clicked. The function first gets the id attribute of the clicked tag (which we set to Crunchbase's unique identifier, their 'permalink') and then uses jQuery to execute an Ajax call for JSON, with a callback. The empty return does nothing except prevent the default click action. The enclosing document.ready method makes sure the script won't try to attach the code until after the HTML is loaded.

Part of the reason this site can do everything it does on the client-side is because Crunchbase's API supports JSON callbacks. In general, client-side Javascript can't go willy-nilly fetching things from other sites because of the same origin policy enforced by browsers for security purposes. But if you're trying to pull the data from a site that supports JSON with callbacks, you can easily get data from it.

The getJSON function sends a request to Crunchbase for the VC's data. You can see an example of the raw JSON here. When the data returns it calls the callback function--parseCB--with the JSON as the argument. Note that this happens asynchronously, so if you send multiple calls (as with the vcbar site, when you click one of the subset buttons) the data does not necessarily come back in the order you asked for it. Or, maybe, at all. The callback function gets called once for each set of JSON. You need to think through the implications, in some cases.

Here we're asking for one set of data, so it's easy.  Here's parseCB:

var parseCB = function(jsn) {
   var idx, yr, mo,
       byear=2005, eyear=2011,
       months=(eyear-byear+1) * 12,
       data=[];
      
   for (var i=0; i < months; i+=1) { data[i] = 0; };
  
   if ("investments" in jsn) {
      for (var i in jsn["investments"]) {
         var j = jsn["investments"][i];
         if ("funding_round" in j) {
            yr = j["funding_round"]["funded_year"];
            mo = j["funding_round"]["funded_month"];
            if (!yr || !mo || (mo == "None")) { continue };
            idx = (parseInt(yr)-byear) * 12 + parseInt(mo) - 1;
            if (idx < 0) { continue };
            data[idx] +=1;
         };
      };
   };
   return bchart(data,byear);
};

The first few lines declare the function's variables. They set the beginning year to 2005, the end year to 2011 and then calculate the number of months in that span. Then it creates an array with a zero value for each month.

The function then parses the JSON. Go look at the raw JSON at the link above again, if you want to see what's going on here. First it tests to see if there is an "investments" key in the JSON. If there is an investments key, the corresponding value will be an array with an entry for each investment. Each entry in this array will be a dictionary with keys for "funded_year" and "funded_month". parseCB first tests to make sure that neither the year nor the month is empty and that the month is not "None", then computes how many months from beginning of 2005 (byear) until the investment was made. It then increments the array element representing that month.

When it is finished slotting each investment into a month, it calls bchart, the charting function.

The Javascript, part 2: Charting

The bar chart function is essentially cribbed from Mike Bostock's bar chart tutorial. It uses the d3.js data manipulation library to create a SVG element in the HTML.

Here's the code, broken into chunks so I can explain it.  It's all inside a

var bchart = function (data, byear) {
   ...
};

First, let's set up some variables. h is the height, totw is the total width, w is the width of each bar, lgst is the largest value in the data to be charted, tks is the number of horizontal ticks we want, years is an array of years from the beginning year (byear) to the end year (this is used to label the x-axis.)

y is a special d3 function that maps the 'domain' to the 'range'. In this case, it maps a value from 0 to lgst to the range 0 to h. That is, y(x) = x * h / lgst. This scales the bars so the largest value in the data is the height of the chart.

var h = 300,
    totw = 800,
    w = totw / data.length,
    lgst = d3.max(data),
    tks = Math.min(lgst,5),
    years = d3.range(byear,byear+data.length/12+1),
    y = d3.scale.linear()
          .domain([0,lgst])
          .range([0,h]);

Then, let's get rid of any chart that happens to already be there, so we don't keep adding new charts one after the other.

$(".chart").remove();

Now we add a SVG element to the div with id="barchart". We will make it wider than totw and higher than h so we have room to add the axes and their labels.

// insert SVG element     
var chart = d3.select("#barchart")
              .append("svg:svg")
                .attr("class","chart")
                .attr("width", totw+40)
                .attr("height", h+40);

Then we'll add the x and y-axis ticks, the light gray lines that help us see what the values are. We use a built-in d3 function called ticks, which chooses sensible values for the ticks based on tks, the number of ticks we want. The way d3 works (and I'm not going to explain this in too much depth, you can go to the d3 site for much better explantions) is that it takes an array of data (the data method below the select ), iterates through each item and uses the enter method to put that data into existing svg elements that match the select. If there are not enough existing elements, it appends them, as here.

The below code iterates through each of the ticks generated by ticks and appends a new svg:line with attributes (x1, y1) and (x2, y2). The methods chained after data can have anonymous functions that have access to the data in the array (d) and the index of the data (i). For instance, the y-axis ticks have an x1 of 20 (I've added an offset of 20 to all the x values to accomodate the y-axis labels) and an x2 of totw+20. The y1 and y2 value are trickier. They are both the same (it's a horizontal line) and they both take the d value (where the tick is), scale it using the y function and then subtract that value from h, because the origin of the svg plotting area, the (0,0) point, is in the top left whereas our chart's (0,0) point is in the bottom left.

The text labels do something similar. The y-axis uses the tick value as a string for the text and the dx attribute to move the label slightly before the axis itself. The x-axis uses the array of years we created earlier as labels, and centers them between ticks.

   // create y-axis ticks
   chart.selectAll("line.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:line")
            .attr("class","hrule")
            .attr("x1",20)
            .attr("x2",totw+20)
            .attr("y1",function(d) { return h-y(d); })
            .attr("y2",function(d) { return h-y(d); })
            .attr("stroke","#ccc");

   // label y-axis ticks  
   chart.selectAll("text.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:text")
            .attr("class","hrule")
            .attr("x",20)
            .attr("y",function(d) { return h-y(d); })
            .attr("dx",-1)
            .attr("text-anchor","end")
            .text(String);

   // create x-axis ticks           
   chart.selectAll("line.vrule")
            .data(years)
        .enter().append("svg:line")
            .attr("class","vrule")
            .attr("y1",h+10)
            .attr("y2",0)
            .attr("x1",function(d) { return (d-byear)*w*12 + 20; })
            .attr("x2",function(d) { return (d-byear)*w*12 + 20; })
            .attr("stroke","#ccc");

   // label x-axis ticks          
   chart.selectAll("text.vrule")
            .data(years)
        .enter().append("svg:text")
            .attr("class","vrule")
            .attr("y",h)
            .attr("x",function(d) { return (d-byear) * w * 12 + w * 6 + 20; })
            .attr("dy",10)
            .attr("text-anchor","middle")
            .text(String);

Now we create the data bars. Here we feed the d3 the array of data. For each of the data elements it creates (using enter) a new svg:rect, a rectangle.  Each rectangle has x and y as its top left point and a width and height. The rectangles will also be styled by the CSS, which we'll talk about later on.

    // create bars
    var bars = chart.selectAll("rect")
            .data(data)
        .enter().append("svg:rect")
            .attr("x", function(d, i) { return i * w + 20; })
            .attr("y", function(d) { return h - y(d); })
            .attr("width",w)
            .attr("height", function(d) { return y(d); }); 

And, finally, the x and y axes. The reason we create the ticks first, then the bars and then the x and y-axis is that this is the order of layering we want, ticks at the bottom, bars on top of them, then the axes.

   // create x-axis
   chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",totw + 20)
        .attr("y2",h)
        .attr("stroke","#000");

   // create y-axis               
   chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",20)
        .attr("y2",0)
        .attr("stroke","#000");

Don't forget to include the function declaration before all the chart code and the '};' after it all. Just saying. Also, the javascript should have the functions first, so essentially in the opposite order presented here. I've put all the javascript in one contiguous piece at the bottom*.

That's the chart. After that, the CSS is a piece of cake.

CSS

Nothing fancy here. Put it in a file called vcbar.css in the same directory as index.html.

.chart {
    margin-left: 40px;
    font: 10px sans-serif;
    shape-rendering: crispEdges;
}
           
.chart rect {
    stroke: white;
    fill: steelblue;
}

And that's it. If you put this code into files on your computer and open index.html from your web browser, you should get a chart. Then go and change the code and see what happens, or add lots more code and do something really, really cool. When you do, tweet me, I want to see it.

-----
* vcbar.js, in total:

var bchart = function (data, byear) {
   var h = 300,
       totw = 800,
       w = totw / data.length,
       lgst = d3.max(data),
       tks = Math.min(lgst,5),
       years = d3.range(byear,byear+data.length/12+1);

    $(".chart").remove();

    var y = d3.scale.linear()
             .domain([0,lgst])
           .range([0,h]);

   // insert SVG element      
    var chart = d3.select("#barchart")
        .append("svg:svg")
            .attr("class","chart")
            .attr("width", totw+40)
            .attr("height", h+40);

   // create y-axis ticks
    chart.selectAll("line.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:line")
            .attr("class","hrule")
            .attr("x1",20)
            .attr("x2",totw+20)
            .attr("y1",function(d) { return h-y(d); })
            .attr("y2",function(d) { return h-y(d); })
            .attr("stroke","#ccc");

   // label y-axis ticks  
    chart.selectAll("text.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:text")
            .attr("class","hrule")
            .attr("x",20)
            .attr("y",function(d) { return h-y(d); })
            .attr("dx",-1)
            .attr("text-anchor","end")
            .text(String);

   // create x-axis ticks           
    chart.selectAll("line.vrule")
            .data(years)
        .enter().append("svg:line")
            .attr("class","vrule")
            .attr("y1",h+10)
            .attr("y2",0)
            .attr("x1",function(d) { return (d-byear)*w*12 + 20; })
            .attr("x2",function(d) { return (d-byear)*w*12 + 20; })
            .attr("stroke","#ccc");

   // label x-axis ticks          
    chart.selectAll("text.vrule")
            .data(years)
        .enter().append("svg:text")
            .attr("class","vrule")
            .attr("y",h)
            .attr("x",function(d) { return (d-byear) * w * 12 + w * 6 + 20; })
            .attr("dy",10)
            .attr("text-anchor","middle")
            .text(String);
   
    // create bars
    var bars = chart.selectAll("rect")
            .data(data)
        .enter().append("svg:rect")
            .attr("x", function(d, i) { return i * w + 20; })
            .attr("y", function(d) { return h - y(d); })
            .attr("width",w)
            .attr("height", function(d) { return y(d); }); 

   // create x-axis
    chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",totw+20)
        .attr("y2",h-.5)
        .attr("stroke","#000");

   // create y-axis               
    chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",20)
        .attr("y2",0)
        .attr("stroke","#000");     
};

var parseCB = function(jsn) {
   var idx, yr, mo,
       byear=2005, eyear=2011,
       months=(eyear-byear+1) * 12,
       data=[];
      
   for (var i=0; i < months; i+=1) { data[i] = 0 };
  
   if ("investments" in jsn) {
      for (var i in jsn["investments"]) {
         var j = jsn["investments"][i];
         if ("funding_round" in j) {
            yr = j["funding_round"]["funded_year"];
            mo = j["funding_round"]["funded_month"];
            if (!yr || !mo || (mo == "None")) { continue };
            idx = (parseInt(yr)-2005) * 12 + parseInt(mo) - 1;
            if (idx < 0) { continue };
            data[idx] +=1  
         };
      };
   };
   return bchart(data,byear);
};

$(document).ready(function () {
   $("a").click(function () {
     var vc = $(this).attr("id");
     $.getJSON("http://api.crunchbase.com/v/1/financial-organization/" + vc + ".js?callback=?",parseCB);
     return;
   });
});

Pace of VC investing by subsector

I couldn't sleep last night so I figured I'd see if I could confirm a nagging suspicion about the early-stage VCs I know. About six months ago it seemed like they were slowing down their pace of investing while the corporates and newer super-angels were doing a lot more deals. If this were true it would be an interesting warning sign.

So I downloaded d3.js, pulled out the list of VCs I put together for VCdelta and built a visualizer for Crunchbase data. It's fun to play with*.

Here's a graph of the deals the 150+ VCs have done since 2005, according to Crunchbase. If you go to the site and click "All" at the bottom, you get this, except it's live to add and subtract either VC firms or round types from and you can hover over the bars and see the names of the companies invested in that month**. You can also, if you click the subsets below, see who I included and who I didn't. And then add or subtract to your heart's content.

What looks like a small downturn in 2008 and 2009 in deals done is mainly due to VCs continuing to do later rounds--B and later. I assume many of these were into companies that were already portfolio companies.

Here are all the VCs, but just the rounds tagged Seed, Angel and A.

This makes it easier to see the dropoff in 2008 and 2009. But the low point in early stage investments came later than I thought, in 2009. It had seemed to me that early 2008 was dryer. Also, according to Crunchbase, more early stage deals are getting done now than in 2007.

New York City is on a roll, right? Right. Below are the NYC funds (not NYC deals) and how many early stage (Seed, Angel, A) deals they did.

Compare this to Sand Hill Road:

Sand Hill Road has remained relatively conservative into 2010 and 2011.

Some other VC subsets. I used the top 20 venture capitalists in Forbes' Midas List to create a 'smart money' subset of firms. Here are their early-stage deals. The pronounced uptick from the lows in 2008 and 2009 into 2010 and 2011 are heartening.

I also made a subset consisting of firms that have been around since before the 1980s, the 'old school.'  I assumed that if they've made it this long, they must be doing something right. Their increase in early stage investments, while less pronounced, is also heartening.

Last, the Super Angels. No surprise here.

The one thing these graphs don't do is support my original thesis, VCs are not slowing down their funding of early-stage companies. Interestingly, I found that even the VCs who have flat-out told me they are slowing down their investing are not really doing so: while there's fear in the market, VCs are also clearly seeing opportunities they can't turn down.

-----
* d3.js is awesome. The Yieldbot guys turned me on to it. I'm just learning it, so I know I'm manhandling it something awful, but it's a joy to work with.
** Let's do the usual caveats: Crunchbase data sucks for this kind of thing. It's incomplete, it's biased, it's not very clean or accurate, etc. This is all completely offset by the fact that it's free. If I had a better dataset, I'd use it, but I don't.

@VCdelta released. Whoops.

Josh Kopelman tweeted about my weekend project today, which is still in alpha (at best.) Until yesterday, the only follower was Dave McClure. But he follows so many accounts that I figured he just auto-followed anyone who tweeted '500 Startups.' I was going to wait until I got back from next week's vacation before mentioning it to anyone, but the twitter-feed is open, so... it's been released.

The website is the API: I wrote a script to look at the portfolio pages of VC websites every night and tweet and post new companies that seem to have been added. It's pretty useful in a way, but has some severe limitations.

The biggest is that scraping is inherently fragile. And I'm going on vacation next week and leaving the computer it's running on at home, running it. If it starts to spew garbage on Monday, well... sorry. I'll fix it when I get back.

It is reporting on differences. So when Stickybits changed to Turntable.fm, it showed that First Round added Turntable. When Tremor Media changed its name to Tremor Video, it got reported, etc.

VCs don't always update their portfolio pages in real-time. This is no substitute for Techcrunch (or Crunchbase, even), it's just faster and easier to scan. I've been adding the VCs in drips and drabs over the past few weeks, so there are certainly additions that got missed because of the timing. The web site is pretty cool, it lets you filter by name and date and sort. That will be more useful over time as more deals get added.

The list of VCs looked at is here. If I missed yours, let me know and I'll add it to the to-do list. Unless your site is in flash (cough, Norwest) or has no portfolio company names (ff Ventures, among several others) or doesn't allow bots (yes, oddly enough, there is a site that checks the user agent and sends my script a page with no real content; the robots.txt--as with all of the pages I look at--allows, but the server doesn't. Why?)

Enjoy.

The new adtech is disruptive, and that's a good thing

I was at Luma Partners' Digital Media Summit today. Great event, saw lots of familiar faces. I watched the adtech panel. Of the people I didn't get a chance to invest in, these are four of the smartest: Brian O'Kelley, Joe Apprendi, Michael Barrett, and Mike Leo.

Mike Leo said something that struck me as wrong. He said, roughly, "30% of the media spend is getting spent on the pipes"--by which I think he meant the other panelists' companies--"and that's eating into the creative and the content, where it should be spent."

This is exactly the wrong way to look at it.

The entire process between the maker of a product or service and the user of that product or service--what we call marketing--is friction. This includes the pipes, the ads themselves, and even the content created to wrap the ads. Friction, all of it.

I agree that we should reduce the friction, make things more efficient. But if we give credit to Wanamaker's 50% waste in advertising spend, then the 30% that's now being spent on the "pipes" is a 40% improvement. A 40% improvement over five years is pretty spectacular. But disruptive technologies do that.

Leo is looking at the wrong place--he's complaining about the one area of the marketing process that has actually shown efficiency gains, while giving a pass to creative and content, the areas that have fallen behind.

Valuation for investors

Twice in the past week I have had pre-product entrepreneurs tell me that they were raising seed rounds at an approximately $10mm pre-money. In both cases I had to pass, despite the merit of the management teams. Both companies told me that they have other early-stage investors ready to fill their rounds, and I'm glad. I am generally of the opinion that if an entrepreneur can get a better valuation while still getting value-add investors, then they should.

Plenty has been written recently by venture capitalists about venture capital to help entrepreneurs. Not much has been written to help newer venture capitalists. I think, in a way, this is because VCs don't care so much if those new to the industry succeed or not. If a new angel loses his shirt, well, one less competitor for me down the line.

I don't believe that, though. I think we need more investors. But smart ones, investors who make some money on their investments and so feel confident reinvesting it in a new round of entrepreneurs. Investors, like everyone else, get smarter and more helpful the more experience they have. Smarter investors is better for the ecosystem.

So, valuation.

Valuation in venture capital is tough. The amount of uncertainty between investment and exit is immense. But that doesn't mean that you shouldn't try to pay the right price. Valuing a startup correctly means estimating risk, not contemplating the unknowable. The idea that was briefly tossed around that valuation doesn't matter because a startup either goes big or dies is wrong. Ludicrous, in fact. There's a range of outcomes for every fund. The cliche that out of each ten investments, two are winners, three are failures and five go sideways shows this. The two winners determine whether the fund is an overall winner or loser, but how sideways the five go determines whether it's a good return or a great return. What happens to the second tier of investments matters.  And, for the math challenged, even if startup returns were binary, when you invest in many of them you get a binomial distribution, so expected value matters.

I look at valuation this way. For every company,

• I think about what the expected exit would be if the entrepreneurs were right: if they are right about the problem, about what their customers want, about their ability to execute, right about everything. 
• I figure out how much dilution I expect before the exit.
• I decide how likely it is they are right and multiply by the expected exit to get an expected value.
• I divide by three, because I'd like my investments as a whole to return 3x*. 

This is the post-money valuation.

An example. Company X is an amazing data-driven adtech company. It's going to disrupt some existing companies and if it does it should sell for $400mm in five years. I think, given the risks, that there's a one in ten chance they will succeed.

But I know they will need to raise a Series A to commercialize the product once it's ready, and a Series B to ramp sales once they have product-market fit, and a Series C to expand the product line. The Series A will be 33% of the company, the Series B will be 25%, and the Series C 20%. My stake will be diluted down to 40% of my original ownership.

So my post-money expected value of the company is $400mm * 10% * 40% = $16mm. I would be looking for a post-money of $5mm. If the company is raising $1mm in the seed round, the pre-money valuation would be $4mm.

You can see the difficulty in the $10mm pre-money. If the company is raising, say, $2mm at a $10mm pre, then the expected exit value would have to be $12mm/(10% * 40%) * 3 = $900mm.

Billion dollar exits are the sine qua non of the venture business. But they are rare. Rarer than you think.

I made a list off the top of my head of some 125 business-to-business advertising exits. I may be missing some obvious ones, but there were only a handful of $500mm plus exits in the last ten years, even fewer billion dollar ones (M&A exits, I didn't count IPOs, so I probably undercounted by one or two.) Here are the $500 million and up exits I have.

Company	Acquiror	Price ($mm)	Date
aQuantive	Microsoft	$5,900	May-07
Doubleclick	Google	$3,100	Apr-07
Omniture	Adobe	$1,800	Sep-09
Overture	Yahoo!	$1,630	Jul-03
Digitas	Publicis	$1,300	Dec-06
NetRatings	Nielsen	$817	Feb-07
AdMob	Google	$750	Nov-09
Right Media	Yahoo!	$680	Jul-07
Lending Tree	IAC	$675	Aug-03
24/7 Real Media	WPP	$650	May-07
Rosetta	Publicis	$575	May-11
Razorfish	Publicis	$530	Aug-09

Of the 125 exits, five were more than a billion, seven were between $500 million and a billion, 20 were between $200 million and $500 million, and 15 were between $100 million and $200 million. The rest were sub-$100 million. Remember, these were all exits--companies that didn't make it weren't counted. There's also a bias in the list because I am more aware of the large exits; I would be surprised if I missed too many billion dollar exits but I am sure I missed many $10mm exits. Also note that only a couple of the billion dollar exits here were as straightforward as my model: aQuantive was built through acquisition (and thus had substantially more dilution), Doubleclick had gone through several owners including the public markets, etc.

In fact, all else being equal (a priori, that is) billion dollar exits returned less overall than $500mm-$1bn exits, because there were fewer of them. Exits between $200mm and $500mm probably returned slightly more than $500mm-$1bn exits (also because there were more of them). The $100mm-$200mm range and less than $100mm range each return less than the $200mm-$500mm range**. Here's my estimate of what each of these ranges returned.

Exit Range	Companies	Total Est. Value
$1bn +	2	$3,000
$500mm - $1bn	7	$5,000
$200mm - $500mm	15	$6,000
$100mm - $200mm	25	$4,000
$50mm - $100mm	50	$3,750
< $50mm	100	$2,500

The sweet spot in adtech, the "average" expected exit value, seems to be around $400mm.

Every industry is different, you need to know yours. Make a list of exits over the last ten years, all exits not just the good ones. Then try and figure out how many companies were funded in your industry. This will inform your expected exit values in the success case as well as help you decide what percentage of funded firms get to an exit. Conditions change all the time, of course, but looking at the last ten years will probably keep you reasonably conservative.

-----
* I asked an engineer friend of mine how comfortable he is being in the buildings he helped design. "Pretty comfortable", he said. "You never worry?" I asked. "Look", he said, "There's a lot of math and experience behind choosing exactly how much steel and concrete the building needs to bear its load. I do the work carefully, run the calculations twice and make absolutely sure the answer I am getting is the right one. Then I multiply by three." [Edit: For those for whom anecdote is not analysis, I'll point out that a 25% IRR compounded annually for five years is 3x].

** The list I made is up here. Click on the headers to sort. I think all of the #N/As are sub $50mm exits except the two bolded ones, which I think are ~$200mm exits. [Edit: Wow, the sorting on the linked table was all screwy. Sorry. Fixed it.]

On failing

Don't confront me with my failures, I have not forgotten them.

- These Days, J. Browne

Six years ago. I was working on a startup with a bunch of friends, a big idea and one that got me out of bed every morning, excited to go to work. We had raised money from a Name and some amazing venture investors. We were going to be the next Google. My other gig, a venture fund, had just had the exit that put it over the top: one of the portfolio companies had gone public. Me and my two partners--one active and one silent--had made the fund back and then some. I was going to make more than living wage from the eight years of work that portfolio represented. At home, we had finished furnishing the house, our third child was on the way, and my roses were finally getting some traction.

I was out with a good friend recently. He told me an old college buddy of ours was having some trouble with his career, with his marriage. I'll call him, I said, I should talk to him. No, my friend said, he doesn't want to talk to you right now. He thinks you wouldn't understand. Everything's always worked for you.

Five years ago. Our Name investor had realized that if we stopped trying to grow the business, it would immediately begin making money. This was not what we wanted, we wanted to be the next Google, not a tiny mortgage lead generator. But he wasn't a venture guy, he wasn't an investor at all, he was a buyer of assets. He scared away the outside money, starved the business of cash, and forced us to accept a buyout. I had been pushed out of the company I had helped start. In the entrepreneurial community, there aren't many marks blacker than that.

Meanwhile, my active partner in the venture fund decided that the silent partner should not be paid their share of the fund, contrary to what our contract seemed to my non-lawyer eyes to say, and certainly contrary to what fairness dictated. When I refused to go along with that, he sued me. He also sued the silent partner. The silent partner fought back, and the whole kit-and-kaboodle got locked down. The vast majority of my assets were frozen, being held hostage to some ridiculous legal squabble for some indeterminate period of time.

Not entirely coincidentally, my marriage fell apart at that exact moment.

I had failed, utterly. I had no job, my net liquid assets were approximately zero, my reputation was in tatters, and the one thing in my life I thought was permanent was over. The struts that supported my sense of self all got knocked out pretty much simultaneously. Everything I had, everything I hoped for. Five years ago, today.

In what way are you the same person today that you were twenty years ago? There is no atom of matter in you that was there twenty years ago, there is no piece of you that is the same as it was. The pieces of you, the functioning of you, the pattern of you are all different. Why are you you? What is you? What is it that continues? When the Buddha told us to meditate on death this is the question he wanted us to ask ourselves: what where you before you were born, what are you after you die? In what way are you still you even twenty minutes from now?

I'll tell you an answer, although me telling you no more gives you wisdom than reading a cookbook gives you nourishment: we are a span of links in a chain of causality. My existence now causes my existence a minute from now, and that a minute from then. There is no me except my self causing my next self every tick of the clock. In consequence, I am subject to the Markov property: how I arrived at the current me is irrelevant, every option available to me is embodied in who I am now, not in how I got here.

I'm no saint and failure did a number on me. Someone told me what intense stress would do to my psyche, the stages of irrationality I would go through. Frustratingly, the knowledge of them did not allow me to avoid them. But failure taught me some humility, and being humble forced me to abandon the personas I had built and my arrogance of idealism. And I finally learned that except in how it hobbled me, my failure made no difference--all that mattered was what I did next.

Five years after everything fell apart I have the best portfolio of startup investments in one of the hottest spaces in tech. People I respect and admire recommend me to entrepreneurs. I am allowed to be productive. And I spent Memorial Day with my loved ones and was happy and relaxed and even looking forward to getting back to work after a long weekend. I appreciate it, all of it, every time. I haven't forgotten I had nothing. I take nothing for granted. But I don't now and never did believe that the past determines the future. The future is determined only by the choices you have now--the ones you can find a way to allow yourself--and what you do with them.

I don't think saying that everything's always worked for me is wholly accurate. But it may be that it is entirely true.

Neu VC

Put up a website. Took down the robots.txt today. Thought I'd tell you all first.

Now before you inform me--as Josh Reich did when I asked him to take an early look--that I should get a new designer, I'll let you know that I designed it myself. With help from my five year old.

Between being an investor and being an entrepreneur, the grass is always greener. I meet plenty of entrepreneurs who want to be VCs. I always ask them why. I love helping people start companies, but it's not the same as starting a company yourself.  That said, I think I add more value investing that I do founding, so investing is what I plan to do for the next fifty years of my professional life (or as long as anyone will let me, whichever comes first.)

While I have resisted describing myself as an 'it' rather than a person, it's definitely still true that people attribute more permanence to an entity than an individual. Thus Neu Venture Capital, where 'we' invest. We is just me (ever since Softbank hired my awesome intern away from me by offering him actual money to do the work rather than just scintillating conversation.) Consider it the royal We, without the royal part. The Neu came from a conversation with a friend in which she insisted that my children preface everything they like with a subtle 'neu.' I scoffed at this until one morning when my youngest informed me that the White House was where President Neubama lives. I will not require all future investments to prefix their company name with neu, but I won't promise it won't influence my decision either.

I looked at a lot of VC sites while figuring out how I wanted it to look, and the one thing that was an absolute requirement was something I learned from my Omnicom days: the operating companies are what it's all about. So the home page--in fact the only page--is the companies I've invested in. There is a box of information about me (with my real picture, not my avatar: bonus!) and an entirely uninformative box about what I am looking to invest in (I'll work on it) but it's primarily about the companies I've invested in. I'm good with that in lots of ways. But mainly because if you're known by the company you keep, I'm in great company.

The client does not care if you are intellectually stimulated, Kendall, they just want the goddamned sales curve to start moving up

Every revolution has its reactionaries. I was going to respond to Kendall Allen's article The Math State, but didn't, for a few reasons: it wasn't convincing enough to need a response, I've stated my answer to the objection elsewhere, and Joe Zawadzki is a better writer than I am.

I want to note, though, that when I accepted an offer to work at Omnicom, some 15 years ago, I was handed a copy of Peppers and Rogers.  This was the future, I was told, One to One Marketing. Some six years later the Economist wrote that "the marketing department is the last part of the modern corporation to resist automation." It's interesting that after so many years of wishing for it, the industry starts to object as soon as it becomes plausible.

This new math state is not new. It is a stage in a journey that some of us have been taking for more than a decade and one that will take another decade to reach its apotheosis. For those who don't like it, I'll trundle out the old Rosser Reeves quote that everyone in the agency world says they believe but that few really do:

What do you want out of me? Fine writing? Do you want masterpieces? Do you want glowing things that can be framed by copywriters? Or do you want to see the goddamned sales curve stop moving down and start moving up? 

- Reality in Advertising, R. Reeves, 1961.

Almost a license to print money

Advertising, in time, proved almost a license to print money, and the effects on broadcasting of the revenue model it introduced can scarcely be overstated. It gave AT&T, and later the rest of the industry, an irresistible incentive not just to broadcast more but to control and centralize the medium. To see why, compare the older model: When revenues came from the sale of radio sets, it was desirable to have as many people broadcasting as possible--nonprofits, churches, and other noncommercial entities. The more broadcasters, the more inducement for the consumer to buy a radio, and the more income for the industry. But once advertisements were introduced, radio became a zero-sum game for the attention of its listeners. Each station wanted the largest possible audience listening to its programming and its advertisements. In this way advertising made rivals of onetime friends, commercial and nonprofit radio.

This is Tim Wu in The Master Switch*, describing the beginnings of commercial radio broadcasting.  When AT&T (who was competing with RCA to dominate radio broadcasting) started using its 'long lines' to carry programs to transmitters across the country, they discovered that advertising to a mass audience was far more profitable--and could support professionally produced, higher quality content--than any other business model available to them. This, in turn, lead the industry to successfully lobby the government to allow only a few, high-power 'clear channel' broadcasters (instead of allowing many lower-power ones.)

It's interesting to see how advertising, because it pays for attention, a resource too easily divisible, caused media to agglomerate. The media industry, in protecting itself from competition, has to limit the number and variety of voices that are heard.  Mass media is, by its nature, homogeneous media.

Two things:

1. Mass media, as it is today, is not the only way things can, or should, be. There was a vibrant radio culture before it became a mass medium, with a more democratic voice.

2. Our adtech allows advertisers to reach small audiences. The advertiser no longer needs mass media. Mass media will, however, fight to maintain their current market position. In almost all of the other media this book chronicles, the government was eventually enlisted to regulate out weaker players. The fight over net neutrality was one of these efforts, but certainly not the last.

-----
* Required reading. This is the first book in twenty years that, as soon as I finished it, I started reading again. A history of the rise, consolidation and disruption of the telephone, radio, motion picture, and television industries, it elucidates the history that can inform scenarios of the possible future of our industry better than any other analysis. It's a good read, too, with wonderful descriptions of the people behind the inventions and companies and how the culture of their times influenced them.

Advertising, the Fallacy of Perfectibility, and the Best Minds of My Generation

Between men and brutes, there is another very specific quality which distinguishes them, and which will admit of no dispute. This is the faculty of self-improvement... perfectibility... It would be melancholy, were we forced to admit that this distinctive and almost unlimited faculty is the source of all human misfortunes; that it is this which, in time, draws man out of his original state, in which he would have spent his days insensibly in peace and innocence; that it is this faculty, which, successively producing in different ages his discoveries and his errors, his vices and his virtues, makes him at length a tyrant both over himself and over nature. 

- The Social Contract and Discourses, Jean-Jacques Rousseau

An entrepreneur I'm working with on an adtech company asked me recently "does any of this make any difference?" It's a good question.

Here's something you probably didn't know about me: when I ran venture capital for the world's largest advertising agency holding company back in the '90s, I refused to invest in internet advertising companies*. I did not think banner advertising was useful. I have not come so very far from that view.

* * * * *

There is a sometimes infuriating disdain of advertising among those who wear Northface and Converse, drink Starbucks and Horizon Organic milk, and find out about the farmers' market from drop cards in the local organic grocery store. But that they can hate advertising while being avid consumers of it isn't irrational, because advertising has two faces.

Before World War I in America, a belief in perfectibility was widespread.  Not just human perfectibility, but technological. Businessmen created and grew businesses based on a belief in perfectibility. All success required was creating the best product at a fair price, driving competitors out of business and cementing a permanent monopoly, all for the betterment of society. Competition, to people like Henry Ford and Theodore Vail, was a waste of resources. Creative destruction would have seemed like madness.

In this no-nonsense model, advertising was product news, an attitude typified by the Bates agency in New York in 1904, who viewed anything more as "bad copy--too literary and too general, lacking prices and good information." John E. Kennedy, the head copywriter for Lord & Thomas in those years thought that "instead of general claims, pretty pictures, or jingles... an ad should offer a concrete reason why the product is worth buying."  One of the students of this method said this method of copywriting "aimed at an appeal to reason and intelligence rather than the time-honored assumption that the public was a mass of dumb, driven sheep, who could be swayed with mere picture-and-catch words."**

This approach, though it was itself a reaction to an earlier character-and-jingles driven approach (Aunt Jemima, the Uneeda slicker boy and the Arrow Shirt man were all products of this earlier approach) finds its intellectual rationalization in the Chicago School. George Stigler described it: advertising provides information to buyers, allowing them to make better choices.  Because the Chicago school believed that economic actors were rational, and that economic activity must have some underlying sense, this was the only coherent explanation... even if the facts on the ground at the time did not bear it out (Stigler published his seminal Economics of Information [pdf] in 1961.)

By that time, advertising had changed. Many times. The War to End All Wars itself may have been the catalyst. At the beginning of the conflict, fact-based ads played well to a righteous population. But later, when war-weariness had taken hold, patriotic advertising showed that appealing to the emotions may be even more powerful, especially when the facts aren't all that pretty. Advertising can persuade. Posters of Uncle Sam pointing his finger and saying "I Want YOU for U.S. Army" worked.

But the creator of that image, James Montgomery Flagg, had misgivings: "A number of us who were too old or too scared to fight prostituted our talents by making posters inciting a large mob of young men who had never done anything to us, to hop over and get shot at... We sold the war to youth."***  These are the poles advertising swings between: providing information so people can make better decisions about which products to buy, and persuading people to do things they might otherwise not have done. This is why people can hate advertising while listening to it closely, both of these approaches are at work, sometimes in the same ad.

The carnage of WWI had another effect: the idea of perfectibility itself fell out of favor. There was no one after that war that could still believe that those in power were infallible or even that doing the right things leads to the right outcomes. A belief in the universal power of reason now had a powerful enemy in American culture: pluralism. I'd argue that the battle between openness and control that we see happening to the internet now (cf. Google v. Apple) is a battle between perfectibility and pluralism, between one person knowing what is best and the belief that progress proceeds from chaos.

* * * * *

Productivity growth has made our lives enormously better. And it is one of the keys to making the lives of our offspring better than ours. If there is one guiding principle in my work, it is trying to help nurture innovations that create productivity growth.

Productivity is not just doing more with ourselves, it's also preventing waste, and guiding resources to their best use. This last is what Hayek called "the economic problem." Hayek said that the solution was the price system, the "invisible hand" that put resources where they are needed. This is an astonishing insight, but it falls short. Hayek, I have to assume, never bought a car.

The price system may be the best way to make sense of decentralized knowledge when the resource being apportioned is tin, as in Hayek's example. But when we go to buy a car, we have many choices at any given price point. The cars, although similar in most regards, are different, and we can't choose on price alone. Choosing the car that is best for us requires more information. Some of this information (mpg, how it looks) we can find out ourselves, if we know to look. But in many cases putting the onus on the potential consumer to discover all possible choices is inefficient. When looking for a place to buy produce, I will not know to look for the farmers' market that I do not know exists. Advertising is key here.

I believe that after an efficient price system, an efficient matching of people with the provider of a product or service that is most appropriate to them is the economic problem.  When Jeff Hammerbacher laments that "The best minds of my generation are thinking about how to make people click ads," I think he's falling for a technocratic idea long discredited, one that only engineers still believe: perfectibility.  Jeff is young, and very smart. He may believe that there is a single best product or service in each category that can be found using the appropriate Google search. And perhaps he believes that he himself should be the person who can decide what people really want, as Theodore Vail did, or Henry Ford tried to do, or Steve Jobs tries to do. Because if he didn't believe in his own omniscience, and if he personally would prefer not to have some other person making his choices for him, then he would have to agree that each person needs information to make the right choices for themselves. In that case, if I put an ad in front of someone, and that person decides to click it, then I must be solving a very difficult problem indeed.

Getting a click means that you have shown the clicker something valuable to them. If you don't believe in perfectibility--in making peoples' choices for them--then the click is a win.

* * * * *

I make a sharp distinction in my ad-tech investing between better matching and persuasion.  I only invest in the former (although sharp lines can not always be drawn.) The data-driven media buying and ad placement companies that are a large part of my portfolio are all clearly aimed at putting the appropriate information in front of the right person****.  I see companies that I think will be successful persuaders, but I don't invest in these; they make me uneasy. I suppose you could say that I'm not very businesslike in that regard, that I don't appreciate the art of selling, that I don't appreciate the art of advertising, that I don't appreciate the cultural value of advertising, that I don't have a sense of humor. I guess that's all true.  But, in my opinion, the companies that make the biggest difference to their customers are the ones that make their lives better.  While I recognize that the customer of the ad-tech company is the advertiser, not the potential consumer, and that the advertiser's life is certainly better if it sells more product, I'm not buying it. While being once-removed from treating the consumer like a mark may ameliorate the ethical considerations, it does not make it good business, not in the long-term.

[Edit, five hours later: I will revise something I wrote forever. So, on the blog, I try to finish and then hit publish. This time, as soon as I did, I realized I had not closed the loop on the argument.

Here's my point: if you believe in perfectibility, that your product is the best one for everybody, you probably also believe that all you need to do is buy mass media ads telling everybody about the existence of your better mousetrap. If you have lost the belief in the perfect, then you will feel the need to compete for the attention of people on other than the facts, because the facts don't differentiate you. Competing on facts is better for society, and for the regular person who sees ads.

The people who are thinking about how to get people to click on ads are of two types: those who just want you to click on the ad, and those who want to put an ad in front of you that would be valuable for you to click on. Saying that the latter is somehow of ephemeral benefit is misunderstanding what is economically worthwhile. By matching the right ad to the right person, we can provide information even while allowing everyone to have their own individual tastes.]

-----
* OK, that deserves some caveats.  I did invest in an ad network... two weeks before their IPO. I sold in the IPO. There was free money like that in 1999. I also heavily invested in the interactive agencies but, and this is my point, none of them would stand for being characterized as purveyors of advertising or, in many cases, any sort of marketing at all. Jeff Dachis' infamous difficulty in describing what Razorfish did on 60 Minutes being a case in point. It wasn't until Agency.com acquired iTraffic that any of the companies I worked with became involved with display advertising in any meaningful way.
** The Mirror Makers: A History of American Advertising and Its Creators, pp. 49-51.
*** Ibid, p.76.
**** While these technologies can also be used to put a persuasive ad in front of the person most liable to be persuaded, I don't think this use provides as much lift, and I hope it will be squeezed out.