OpenRTB and Architectural Innovation

I sent out a good number of emails on Sunday asking for opinions on OpenRTB.  People were mainly dismissive, partly because of the secretive way in which it was concocted, partly because some think Google is better accomodated than challenged, but mainly because the current goals of the spec are slight.  After reading the spec, I was a bit underwhelmed myself.  But I've changed my mind.

In a post in April, I talked about architectural considerations in ecosystem design, taking the engineering concept of End-to-End as an analogy.   In that post I was pushing for architectural change because innovation is highly dependent on a layered and modular architecture*.  Architecture is important.

But architectural change is also interesting in determining winners and losers, and that was my deeper motivation.  From a (technically astute) businessperson's point of view, the article to read is Rebecca Henderson and Kim Clark's Architectural Innovation: the Reconfiguration of Existing Product Technologies and the Failure of Established Firms.  They point out that "architectural" change favors innovators over incumbents.

In laying out their thesis, the authors talk about the failures of incumbent firms in adapting to relatively minor market changes, despite their deep expertise in the core components of the new products being built**.  They also make the distinction between incremental change, radical change and architectural change to draw attention to the fact that seemingly minor changes in the architecture of a system are actually more likely to cause incumbent dislocation than radical changes in the underlying technology.  This is an extremely important point.

The essence of architectural innovation is the reconfiguration of an established system to link together existing components in a new way... Architectural innovation is often triggered by a change in a component... that creates new interactions and new linkages with other components in the established product...

Established firms often have a surprising degree of difficulty in adapting to architectural innovation. Incremental innovation tends to reinforce the competitive positions of established firms, since it builds on their core competencies... In contrast, radical innovation creates unmistakable challenges for established firms, since it destroys the usefulness of their existing capabilities...

Architectural innovation presents established firms with a more subtle challenge... established organizations require significant time (and resources) to identify a particular innovation as architectural, since architectural innovations can often initially be accomodated within old frameworks.  Radical innovation tends to be obviously radical--the need for new modes of learning and new skills becomes quickly apparent... the introduction of new linkages is much harder to spot.  Since the core concepts of the design remain untouched, the organization may mistakenly believe that it understands the new technology***.

Dismissing OpenRTB as not being really anything very interesting at all is missing the point.  OpenRTB is not radical change, it barely qualifies as incremental change.  But it is architectural change.  It is the reconfiguration of existing linkages, or the beginning of it

Architectural change is subtle.  It is ignorable, for the time being.  But it may--may--be enough to change the existing architecture, the architecture that is more and more contained within Google.  Those that realize this and adapt to it will prosper.  Those that dismiss it--either because it is too subtle or because they are a large incumbent and ignore those who profess to compete with them--might find themselves Xeroxed.

There's a subtle belief in our VC-backed community that technological innovation is the sine qua non, the ne plus ultra.  Build a better mousetrap and the world will beat a path to your door.  Unfortunately, we're wrong.  Build a better technology and the incumbent will copy your innovation: they will notice a better technology pretty quickly.  What we need to do is change the competitive dynamic by shaping the architecture of the ecosystem.  This is something we can do without the permission of the Incumbent, and something the Incumbent will have a hard time responding to.

OpenRTB is just as start.  But the more new linkages we can create in the ecosystem, the better chance we have to compete based on merit.

-----

* An overview of this argument and its consequences was brought to my attention by Brad Burnham somewhat after I wrote about it: Internet Architecture and Innovation.  I want to say that this book is really excellent, because it looks like it, but I haven't had the time to do anything but flip through it yet.
** I am not in love with their chosen examples.  In both cases the lower-cost/lower-functionality disruptive (as in Innovator's Dilemma disruptive) element is also present, so as natural experiments they leave something to be desired.  Christensen, in fact, draws heavily from Henderson's work in his book--IMHO, more heavily than he seems to admit.  His description of this paper, in Dilemma, mentions their thesis primarily as a study in organizational structure.  Christensen then talks about "value networks" instead of architecture--and Dilemma is an essential read--but I think Henderson and Kim's work is more directly to the point here.
*** Read the article, really.  The case studies alone are worth it (and might give some comfort to those of us who despair at the ever-incipient chaos in our little sub-industry by highlighting that this is not something we managed to invent ourselves but is, in fact, normal.)

Old Style vs. New Style VC

Wrote a comment over on David Lerner's blog on his "Are Super-Angels Extinct" post. He said

What I am saying is that some of these superangel funds may structurally resemble traditional VC funds, but they are something altogether different- and more akin to an angel group.

My comment was that structure matters.  If you set up the Red Cross just like a bank, with the same incentives, they would have caused the Panic of 2007.

Maybe it's just two different world-views.  I've thought about this and written about it for a long time.  My take is that if you want to create a better venture capital system, you need to change the system, not just the people.  Calling yourself a super-angel and saying you are different does not change the investing world.  Changing the processes and structures of the investing world will change the investing world.  There are firms (and super-angels) doing this.

In the spirit of changing the game, some "old-style" and "new-style" comparisons:

Old-Style	New Style
Money managers	Company builders
Organized like a law firm	Organized like a start-up
Generalists	'T-shaped'
Managing $1bn	Managing $25mm
Living on management fees	Living on expected future carry
Scaling by hiring more partners	Scaling by being more efficient
"Loose lips sink ships"	"You should read my blog"
Going to NVCA meetings	Going to the R Meetup
Joining Angel Investor groups	Writing $25k checks
Starting an owned seed fund/ incubator/ hackathon	Supporting a grass roots effort
Sports jackets	Kicks
Networkers	Community builders
Lawyers	Series Seed
Participating preferred	Straight preferred
51%	15%
LPs measure success by: IRR	LPs measure success by: IRR

Coinvestor Graph Code

I've had a bunch of people asking for the data behind the coinvestor network map and a few asking for the code.  It's actually pretty easy to generate the basic graph from the Crunchbase API.  The hard part was cleaning the Crunchbase data, augmenting it with some other data sources and then making sense of the resulting graph.

But even so, the skeleton code is below. It will get you started. Then take a look at the raw output of Crunchbase's API. Play a bit with NetworkX and note that you can tag any node or any edge with whatever data you want. Then go create your own! If you find anything interesting, let me know.


# Crunchbase to NetworkX network builder
#
# Builds a network from the Crunchbase database and outputs it in graphml format.
#
# Required modules:
#    simplejson (http://undefined.org/python/#simplejson)
#    networkx (http://networkx.lanl.gov/)

import urllib2, simplejson as json, networkx as nx

def getCBinfo(namespace, permalink):
    api_url = "http://api.crunchbase.com/v/1/%s/%s.js" % (namespace, permalink)
    return json.loads(urllib2.urlopen(api_url).read())

def add_clique(G,investors):
    # Take a set of investors and add them to the graph, along with edges
    # between them all. Where an edge already exists, increment its weight.
    l_inv = len(investors)
    if l_inv > 1:
        # add nodes
        for inv, typ in investors:
            G.add_node(inv, inv_type = typ)
        # add edges
        for i in range(0,l_inv-1):
            for j in range(i+1,l_inv):
                if G.edge[investors[i][0]].has_key(investors[j][0]):
                    G.edge[investors[i][0]][investors[j][0]]['weight'] += 1
                else:
                    G.add_edge(investors[i][0],investors[j][0],weight=1)
    return G


# Main.
# Get the list of companies Crunchbase has data on
company_names = json.loads(urllib2.urlopen("http://api.crunchbase.com/v/1/companies.js").read())

# initialize Graph
G = nx.Graph()

# Iterate through companies, getting CB data on each
for company in company_names:
    try:
        co_info = getCBinfo('company', company['permalink'])
    except:
        continue

    # For each company make a set of all investors
    investors = set()
    if co_info.has_key('funding_rounds') and co_info['funding_rounds']:
        for iround in co_info['funding_rounds']:           
            for investment in iround['investments']:
                for i_type in ['financial_org','person','company']:
                    if investment[i_type]:
                        investors.add((investment[i_type]['permalink'],i_type))

    # Add investors and edges between them to the graph
    G = add_clique(G,list(investors))

# Write the network to a graphml file /projects/cb_graph.graphml
# NetworkX supports many other formats as well, check the docs.
nx.write_graphml(G,"/projects/cb_graph.graphml")


Co-evolution and other housekeeping

A couple of weeks ago I wrote something for AdExchanger's 'Predictions for 2011' series. It needed to be brief, and I wanted to talk about what entrepreneurs will be beta testing two years from now.  I made a couple of observations that are pretty obviously true and then ventured this prediction:

Towards the end of [2011], the smartest entrepreneurs will start thinking about how to reinvent the core platforms to better support the needs of the best emerging applications. A dynamic similar to the software/hardware co-evolution of the '70s and '80s will begin, creating similar strategic opportunities.

I actually think this is a pretty tame thought, but once John published it people starting asking me what I meant and how it's actionable.

***** 

I should talk about what I'm thinking generally, though.  I've been pretty quiet since the Summer and where I used to be tightly focused on the data-driven web-based display-ad exchange ecosystem, I'm now a bit broader*.

1. In web-based display I've continued my 2010 push on the publisher side of things.  In addition to my investment in MetaMarkets--who are doing some pretty astounding things--and advising PubGears, I've got another pub side investment that should be announced anon.  I'm still looking for others doing something unique here, but to some extent I feel like these three companies are building 80% of what's undone in getting publishers back their negotiating leverage in a data-driven world.

2. The only other pieces of the web-based ROI feedback loop that I articulated in February that remain unaddressed are the piece at the marketers and the piece at the individuals.  I am still actively looking for companies in the former.  I've met a couple of really exciting ones and am working with them to get to launch-readiness, but want to meet more; this is a big area.  I'm trying to figure out what, if anything, could work at the latter.  I would have invested in Hunch, had I the opportunity back then, but I think there are other ways to address the consumer side of the people-product matching problem.

3.  Social may be one.  The social loop will share superficial characteristics with the display loop, but it's really completely different.  A softer, more subjective approach is needed. IMHO, the area with the most near-term leverage will be tools that help communicators understand the impact of how they are communicating and then help them make better decisions.  I've made an investment here that will probably also be announced anon.  I'm treading carefully in other areas of social because I've seen a lot of ideas imported from the display ad world rise and plateau: social is different and harder to scale.

4. Mobile.  In addition to my long-ago investment in Pinch Media (now Flurry) I've made an investment in a geo-data startup.  This one has not announced publicly, but it's a big idea.  Other than that, again I'm treading carefully.  Steve Jobs' reaction to Flurry earlier this year is an example of why: no matter where you invest in mobile, you're at the mercy of some pretty ruthless gatekeepers.  That said, it's exciting to think about how much value startups could add through the mobile platform.

5. Data is a big problem.  And, to be clear, it's been a big problem for a long time so it has some big and expensive solutions sold by big and slow corporations.  But the companies that are bringing data-munging solutions into the reach of smaller companies operating without teams of specialists are pretty interesting.  I've been trying to get a feel for the process, on a small scale, as my last couple of posts show.  But I've got a lot to learn before I even begin to feel competent.  (If you have something to teach me, I'll buy lunch.)  I want to find companies that can democratize data science.

6. Respect for the individual.  I think about this a lot.  I don't talk about it a lot.  It's a difficult area and using a rational thought process on something that's pre-rational in our makeup is tough.  I haven't been able to organize my thoughts, so I have no idea how I support the value-creation process.  But it's always there in the back of my mind.  I am open to suggestions.

So that's what I've been doing and thinking.  Points 2 through 5 will probably continue to be my focus through most of 2011.

*****

In terms of the AdExchanger bit, here's what I was getting at.

At IBM in 1988, in meetings to discuss moving an instruction's execution from the microcode engine to the hard-coded execution engine, I realized that our design team was at the tail-end of a long chain of product-market fit interactions. Because of the way end-users were using applications, applications had changed which parts of the operating system were critical paths, and the operating system designers had come back to us hardware folk asking that certain instructions be optimized for performance.

This was part of a long-standing and ongoing co-evolution between the hardware and the software.  Innovations on the hardware side changed what software was viable.  Innovations on the software side changed what was needed of the hardware.  Read Melinda Varian's intensely interesting VM and the VM Community, where she talks about the development of IBM's time sharing operating system, and think about it as a co-evolutionary process.  IBM in 1964 opened an office in Cambridge as a liaison to the MIT software engineers developing the first time-sharing operating systems.  Because of its proximity to the users, this office was instrumental in pushing IBM to change its mindset from building machines optimized for batch processing to building machines optimized for time-sharing.

During the next few years, both sides--the operating system writers and the hardware designers--pushed the other side to optimize around what they thought was needed or what they could deliver.  The software engineers pushed for address relocation capability and other features needed for time-sharing. The hardware engineers spec-ed a new processor to meet those requirements.  The hardware engineers floated the idea of virtual machines.  The software engineers built an operating system to take advantage of them**. 

Calling this co-evolutionary may be oversimplifying.  In fact, each layer of the stack co-evolved with the adjacent layers.  Sometimes the tension between different layers evolving differently led to entirely new organisms forking off, like Multics (which then evolved into Unix and its descendants.)  This process can happen in any multi-layer ecosystem that has different actors in different layers.

The data-driven display ecosystem is like that.  The tensions between co-evolving layers is evident (AppNexus/Google anyone?)  And forks are emerging.  I expect that within three years there will be a major fork away from the owned exchanges into a crossing platform that can better support the demands of the adjacent layers: the data exchanges, analytics, the DSPs and the SSPs.  Right now none of these are especially happy with what the exchanges are offering.  Not to say that they're unhappy, but they each have a laundry list of things they would improve or change.  The best of them are creating ways to avoid the exchanges, but only because there is no good alternative.  A platform would still be most efficient.

The exchanges, on the other hand, seem to be pretty content with the way things are.  Or, at least, they feel that they should be controlling the pace and direction of the evolution.  This opens the way for entrepreneurs to build something disruptive.  If you're an entrepreneur with the chops to build something here, you should start thinking about it soon.  2012 will be too late.

-----
* No Thanksgiving jokes, please.
** One great anecdote talks about how the early CP/CMS OS could only keep one virtual machine in memory at a time.  So when a user logged on, the OS would reserve space on the paging drum for the copy of their virtual machine while somebody else's was running.  When the paging drum was full, space would be reserved on disk.  Since the paging drum was so much faster than disk, people started showing up to work earlier and earlier so they could get a slot on the drum.  Finally the OS designers made the page slot allocation mechanisms dynamic.  They just weren't morning people.

More VC Coinvestment Visualizations

I've always believed that unless you try your hand at something, you can't really appreciate the people who do it well.  That was my motivation in creating the VC Coinvestment Network Map I posted two weeks ago.  And believe me, after munging that together I did appreciate the complexity of the process and the expertise of the people who can do it well.

As a side-benefit, I got to meet and talk to several people in the data visualization/network analysis community.  Drew Conway over at Zero Intelligence Agents was the first, and he put up a visualization of the data that teased out some of the structure that I couldn't find.

Now Linkfluence has put up a visualization that does something different.  It doesn't find the 'bones' of the data, as Drew did, but it allows you to find nodes and interact with them.  Below is a screen shot of my node and its neighbors (i.e. the companies and people I have coinvested with, per Crunchbase and AngelList data as of two or three weeks ago*.)  The screenshot doesn't show my cursor hovering over KP, but that's why their name shows up.  Also, we added links back to the Crunchbase database from each node (some of the people don't have CB entries, but that's a small minority of the nodes.)

It's pretty cool to play with.  Take a look.


-----
* The caveats and filtering I did to the data noted in my previous post still apply.