Thursday, August 4, 2011

How I Wrote VCBar

All the people ask me
How I wrote elastic man.

                     - The Fall
My friend Chris Wiggins asked me to post the code for the VC bar chart generator I blogged earlier this week. It's here.

It's an interesting project if only because it's run entirely on the client-side. There's no server side (except, of course, for delivering the files to you.) This is possible because the Crunchbase API supports JSON callbacks. Every bit of code in the git repo is as you see it on http://neuvc.com/labs/vcbar.

But in the spirit of making the source code available, I'm going to go one better and show you how to write your own visualization of Crunchbase data. Because there's no server-side, you can play with this code on your computer with nothing more than a text editor and a web browser.

Adapt this code to visualize other data sets: people respond to visualizations and, as this shows, it's not very hard to make them.

This code is going to be as bare as possible, no bells and whistles. I hope to illustrate just the bones of it. You can add bells and whistles and DTD declarations to your hearts' delight, but this works too.

*****

The program will be broken into three parts: the HTML, the CSS and the Javascript.

The HTML

Create a directory on your computer, download d3.js from https://github.com/mbostock/d3/archives/master, unzip the archive and move the file d3.js into your new directory. Then create a file named index.html in the directory. Put this in it:

<html> 
   <head>  
      <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.2/jquery.min.js"></script> 
      <script src="d3.js"></script>  
      <script src="vcbar.js"></script> 
      <link rel="stylesheet" type="text/css" href="vcbar.css" /> 
   </head> 

   <body> 
      <div id="barchart"> 
      </div> 
      <div id="controls"> 
         <a href="#" id="union-square-ventures">Union Square Ventures</a>
         <a href="#" id="true-ventures">True Ventures</a>
       </div> 
   </body> 
</html>

Pretty simple. The head loads the javascript (including jQuery from Google's CDN) and the CSS. The body has two divs, one named "barchart"--this is where the javascript will put the chart object itself--and one named "controls", where links for the two VC firms this example will link to will live. Note that the links do not link anywhere. We will use the javascript to execute an action when a link is clicked.

The Javascript, part 1: Getting and Parsing the data

Put all the javascript into a file called vcbar.js in the same directory as index.html.

There are three things we want to do in the program:
1. Detect when one of the links is clicked;
2. Get and parse the data;
3. Display the bar chart.

The first is easy, especially using jQuery:
$(document).ready(function () {
   $("a").click(function () {
      var vc = $(this).attr("id");
      $.getJSON("http://api.crunchbase.com/v/1/financial-organization/" +
               vc + ".js?callback=?",parseCB);
      return;
   });
});

This code uses jQuery (the '$') to run an anonymous function every time an <a> tag is clicked. The function first gets the id attribute of the clicked tag (which we set to Crunchbase's unique identifier, their 'permalink') and then uses jQuery to execute an Ajax call for JSON, with a callback. The empty return does nothing except prevent the default click action. The enclosing document.ready method makes sure the script won't try to attach the code until after the HTML is loaded.

Part of the reason this site can do everything it does on the client-side is because Crunchbase's API supports JSON callbacks. In general, client-side Javascript can't go willy-nilly fetching things from other sites because of the same origin policy enforced by browsers for security purposes. But if you're trying to pull the data from a site that supports JSON with callbacks, you can easily get data from it.

The getJSON function sends a request to Crunchbase for the VC's data. You can see an example of the raw JSON here. When the data returns it calls the callback function--parseCB--with the JSON as the argument. Note that this happens asynchronously, so if you send multiple calls (as with the vcbar site, when you click one of the subset buttons) the data does not necessarily come back in the order you asked for it. Or, maybe, at all. The callback function gets called once for each set of JSON. You need to think through the implications, in some cases.

Here we're asking for one set of data, so it's easy.  Here's parseCB:

var parseCB = function(jsn) {
   var idx, yr, mo,
       byear=2005, eyear=2011,
       months=(eyear-byear+1) * 12,
       data=[];
      
   for (var i=0; i < months; i+=1) { data[i] = 0; };
  
   if ("investments" in jsn) {
      for (var i in jsn["investments"]) {
         var j = jsn["investments"][i];
         if ("funding_round" in j) {
            yr = j["funding_round"]["funded_year"];
            mo = j["funding_round"]["funded_month"];
            if (!yr || !mo || (mo == "None")) { continue };
            idx = (parseInt(yr)-byear) * 12 + parseInt(mo) - 1;
            if (idx < 0) { continue };
            data[idx] +=1;
         };
      };
   };
   return bchart(data,byear);
};

The first few lines declare the function's variables. They set the beginning year to 2005, the end year to 2011 and then calculate the number of months in that span. Then it creates an array with a zero value for each month.

The function then parses the JSON. Go look at the raw JSON at the link above again, if you want to see what's going on here. First it tests to see if there is an "investments" key in the JSON. If there is an investments key, the corresponding value will be an array with an entry for each investment. Each entry in this array will be a dictionary with keys for "funded_year" and "funded_month". parseCB first tests to make sure that neither the year nor the month is empty and that the month is not "None", then computes how many months from beginning of 2005 (byear) until the investment was made. It then increments the array element representing that month.

When it is finished slotting each investment into a month, it calls bchart, the charting function.

The Javascript, part 2: Charting

The bar chart function is essentially cribbed from Mike Bostock's bar chart tutorial. It uses the d3.js data manipulation library to create a SVG element in the HTML.

Here's the code, broken into chunks so I can explain it.  It's all inside a

var bchart = function (data, byear) {
   ...
};

First, let's set up some variables. h is the height, totw is the total width, w is the width of each bar, lgst is the largest value in the data to be charted, tks is the number of horizontal ticks we want, years is an array of years from the beginning year (byear) to the end year (this is used to label the x-axis.)

y is a special d3 function that maps the 'domain' to the 'range'. In this case, it maps a value from 0 to lgst to the range 0 to h. That is, y(x) = x * h / lgst. This scales the bars so the largest value in the data is the height of the chart.

var h = 300,
    totw = 800,
    w = totw / data.length,
    lgst = d3.max(data),
    tks = Math.min(lgst,5),
    years = d3.range(byear,byear+data.length/12+1),
    y = d3.scale.linear()
          .domain([0,lgst])
          .range([0,h]);

Then, let's get rid of any chart that happens to already be there, so we don't keep adding new charts one after the other.

$(".chart").remove();

Now we add a SVG element to the div with id="barchart". We will make it wider than totw and higher than h so we have room to add the axes and their labels.

// insert SVG element     
var chart = d3.select("#barchart")
              .append("svg:svg")
                .attr("class","chart")
                .attr("width", totw+40)
                .attr("height", h+40);

Then we'll add the x and y-axis ticks, the light gray lines that help us see what the values are. We use a built-in d3 function called ticks, which chooses sensible values for the ticks based on tks, the number of ticks we want. The way d3 works (and I'm not going to explain this in too much depth, you can go to the d3 site for much better explantions) is that it takes an array of data (the data method below the select ), iterates through each item and uses the enter method to put that data into existing svg elements that match the select. If there are not enough existing elements, it appends them, as here.

The below code iterates through each of the ticks generated by ticks and appends a new svg:line with attributes (x1, y1) and (x2, y2). The methods chained after data can have anonymous functions that have access to the data in the array (d) and the index of the data (i). For instance, the y-axis ticks have an x1 of 20 (I've added an offset of 20 to all the x values to accomodate the y-axis labels) and an x2 of totw+20. The y1 and y2 value are trickier. They are both the same (it's a horizontal line) and they both take the d value (where the tick is), scale it using the y function and then subtract that value from h, because the origin of the svg plotting area, the (0,0) point, is in the top left whereas our chart's (0,0) point is in the bottom left.

The text labels do something similar. The y-axis uses the tick value as a string for the text and the dx attribute to move the label slightly before the axis itself. The x-axis uses the array of years we created earlier as labels, and centers them between ticks.

   // create y-axis ticks
   chart.selectAll("line.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:line")
            .attr("class","hrule")
            .attr("x1",20)
            .attr("x2",totw+20)
            .attr("y1",function(d) { return h-y(d); })
            .attr("y2",function(d) { return h-y(d); })
            .attr("stroke","#ccc");

   // label y-axis ticks  
   chart.selectAll("text.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:text")
            .attr("class","hrule")
            .attr("x",20)
            .attr("y",function(d) { return h-y(d); })
            .attr("dx",-1)
            .attr("text-anchor","end")
            .text(String);

   // create x-axis ticks           
   chart.selectAll("line.vrule")
            .data(years)
        .enter().append("svg:line")
            .attr("class","vrule")
            .attr("y1",h+10)
            .attr("y2",0)
            .attr("x1",function(d) { return (d-byear)*w*12 + 20; })
            .attr("x2",function(d) { return (d-byear)*w*12 + 20; })
            .attr("stroke","#ccc");

   // label x-axis ticks          
   chart.selectAll("text.vrule")
            .data(years)
        .enter().append("svg:text")
            .attr("class","vrule")
            .attr("y",h)
            .attr("x",function(d) { return (d-byear) * w * 12 + w * 6 + 20; })
            .attr("dy",10)
            .attr("text-anchor","middle")
            .text(String);

Now we create the data bars. Here we feed the d3 the array of data. For each of the data elements it creates (using enter) a new svg:rect, a rectangle.  Each rectangle has x and y as its top left point and a width and height. The rectangles will also be styled by the CSS, which we'll talk about later on.

    // create bars
    var bars = chart.selectAll("rect")
            .data(data)
        .enter().append("svg:rect")
            .attr("x", function(d, i) { return i * w + 20; })
            .attr("y", function(d) { return h - y(d); })
            .attr("width",w)
            .attr("height", function(d) { return y(d); }); 

And, finally, the x and y axes. The reason we create the ticks first, then the bars and then the x and y-axis is that this is the order of layering we want, ticks at the bottom, bars on top of them, then the axes.

   // create x-axis
   chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",totw + 20)
        .attr("y2",h)
        .attr("stroke","#000");

   // create y-axis               
   chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",20)
        .attr("y2",0)
        .attr("stroke","#000");

Don't forget to include the function declaration before all the chart code and the '};' after it all. Just saying. Also, the javascript should have the functions first, so essentially in the opposite order presented here. I've put all the javascript in one contiguous piece at the bottom*.

That's the chart. After that, the CSS is a piece of cake.

CSS

Nothing fancy here. Put it in a file called vcbar.css in the same directory as index.html.

.chart {
    margin-left: 40px;
    font: 10px sans-serif;
    shape-rendering: crispEdges;
}
           
.chart rect {
    stroke: white;
    fill: steelblue;
}

And that's it. If you put this code into files on your computer and open index.html from your web browser, you should get a chart. Then go and change the code and see what happens, or add lots more code and do something really, really cool. When you do, tweet me, I want to see it.

-----
* vcbar.js, in total:

var bchart = function (data, byear) {
   var h = 300,
       totw = 800,
       w = totw / data.length,
       lgst = d3.max(data),
       tks = Math.min(lgst,5),
       years = d3.range(byear,byear+data.length/12+1);

    $(".chart").remove();

    var y = d3.scale.linear()
             .domain([0,lgst])
           .range([0,h]);

   // insert SVG element      
    var chart = d3.select("#barchart")
        .append("svg:svg")
            .attr("class","chart")
            .attr("width", totw+40)
            .attr("height", h+40);

   // create y-axis ticks
    chart.selectAll("line.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:line")
            .attr("class","hrule")
            .attr("x1",20)
            .attr("x2",totw+20)
            .attr("y1",function(d) { return h-y(d); })
            .attr("y2",function(d) { return h-y(d); })
            .attr("stroke","#ccc");

   // label y-axis ticks  
    chart.selectAll("text.hrule")
            .data(y.ticks(tks))
        .enter().append("svg:text")
            .attr("class","hrule")
            .attr("x",20)
            .attr("y",function(d) { return h-y(d); })
            .attr("dx",-1)
            .attr("text-anchor","end")
            .text(String);

   // create x-axis ticks           
    chart.selectAll("line.vrule")
            .data(years)
        .enter().append("svg:line")
            .attr("class","vrule")
            .attr("y1",h+10)
            .attr("y2",0)
            .attr("x1",function(d) { return (d-byear)*w*12 + 20; })
            .attr("x2",function(d) { return (d-byear)*w*12 + 20; })
            .attr("stroke","#ccc");

   // label x-axis ticks          
    chart.selectAll("text.vrule")
            .data(years)
        .enter().append("svg:text")
            .attr("class","vrule")
            .attr("y",h)
            .attr("x",function(d) { return (d-byear) * w * 12 + w * 6 + 20; })
            .attr("dy",10)
            .attr("text-anchor","middle")
            .text(String);
   
    // create bars
    var bars = chart.selectAll("rect")
            .data(data)
        .enter().append("svg:rect")
            .attr("x", function(d, i) { return i * w + 20; })
            .attr("y", function(d) { return h - y(d); })
            .attr("width",w)
            .attr("height", function(d) { return y(d); }); 

   // create x-axis
    chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",totw+20)
        .attr("y2",h-.5)
        .attr("stroke","#000");

   // create y-axis               
    chart.append("svg:line")
        .attr("x1",20)
        .attr("y1",h)
        .attr("x2",20)
        .attr("y2",0)
        .attr("stroke","#000");     
};

var parseCB = function(jsn) {
   var idx, yr, mo,
       byear=2005, eyear=2011,
       months=(eyear-byear+1) * 12,
       data=[];
      
   for (var i=0; i < months; i+=1) { data[i] = 0 };
  
   if ("investments" in jsn) {
      for (var i in jsn["investments"]) {
         var j = jsn["investments"][i];
         if ("funding_round" in j) {
            yr = j["funding_round"]["funded_year"];
            mo = j["funding_round"]["funded_month"];
            if (!yr || !mo || (mo == "None")) { continue };
            idx = (parseInt(yr)-2005) * 12 + parseInt(mo) - 1;
            if (idx < 0) { continue };
            data[idx] +=1  
         };
      };
   };
   return bchart(data,byear);
};

$(document).ready(function () {
   $("a").click(function () {
     var vc = $(this).attr("id");
     $.getJSON("http://api.crunchbase.com/v/1/financial-organization/" + vc + ".js?callback=?",parseCB);
     return;
   });
});

Monday, August 1, 2011

Pace of VC investing by subsector

I couldn't sleep last night so I figured I'd see if I could confirm a nagging suspicion about the early-stage VCs I know. About six months ago it seemed like they were slowing down their pace of investing while the corporates and newer super-angels were doing a lot more deals. If this were true it would be an interesting warning sign.

So I downloaded d3.js, pulled out the list of VCs I put together for VCdelta and built a visualizer for Crunchbase data. It's fun to play with*.

Here's a graph of the deals the 150+ VCs have done since 2005, according to Crunchbase. If you go to the site and click "All" at the bottom, you get this, except it's live to add and subtract either VC firms or round types from and you can hover over the bars and see the names of the companies invested in that month**. You can also, if you click the subsets below, see who I included and who I didn't. And then add or subtract to your heart's content.

What looks like a small downturn in 2008 and 2009 in deals done is mainly due to VCs continuing to do later rounds--B and later. I assume many of these were into companies that were already portfolio companies.

Here are all the VCs, but just the rounds tagged Seed, Angel and A.
This makes it easier to see the dropoff in 2008 and 2009. But the low point in early stage investments came later than I thought, in 2009. It had seemed to me that early 2008 was dryer. Also, according to Crunchbase, more early stage deals are getting done now than in 2007.

New York City is on a roll, right? Right. Below are the NYC funds (not NYC deals) and how many early stage (Seed, Angel, A) deals they did.

Compare this to Sand Hill Road:

Sand Hill Road has remained relatively conservative into 2010 and 2011.

Some other VC subsets. I used the top 20 venture capitalists in Forbes' Midas List to create a 'smart money' subset of firms. Here are their early-stage deals. The pronounced uptick from the lows in 2008 and 2009 into 2010 and 2011 are heartening.


I also made a subset consisting of firms that have been around since before the 1980s, the 'old school.'  I assumed that if they've made it this long, they must be doing something right. Their increase in early stage investments, while less pronounced, is also heartening.

Last, the Super Angels. No surprise here.

The one thing these graphs don't do is support my original thesis, VCs are not slowing down their funding of early-stage companies. Interestingly, I found that even the VCs who have flat-out told me they are slowing down their investing are not really doing so: while there's fear in the market, VCs are also clearly seeing opportunities they can't turn down.

-----
* d3.js is awesome. The Yieldbot guys turned me on to it. I'm just learning it, so I know I'm manhandling it something awful, but it's a joy to work with.
** Let's do the usual caveats: Crunchbase data sucks for this kind of thing. It's incomplete, it's biased, it's not very clean or accurate, etc. This is all completely offset by the fact that it's free. If I had a better dataset, I'd use it, but I don't.