Co-evolution and other housekeeping

A couple of weeks ago I wrote something for AdExchanger's 'Predictions for 2011' series. It needed to be brief, and I wanted to talk about what entrepreneurs will be beta testing two years from now.  I made a couple of observations that are pretty obviously true and then ventured this prediction:

Towards the end of [2011], the smartest entrepreneurs will start thinking about how to reinvent the core platforms to better support the needs of the best emerging applications. A dynamic similar to the software/hardware co-evolution of the '70s and '80s will begin, creating similar strategic opportunities.

I actually think this is a pretty tame thought, but once John published it people starting asking me what I meant and how it's actionable.

***** 

I should talk about what I'm thinking generally, though.  I've been pretty quiet since the Summer and where I used to be tightly focused on the data-driven web-based display-ad exchange ecosystem, I'm now a bit broader*.

1. In web-based display I've continued my 2010 push on the publisher side of things.  In addition to my investment in MetaMarkets--who are doing some pretty astounding things--and advising PubGears, I've got another pub side investment that should be announced anon.  I'm still looking for others doing something unique here, but to some extent I feel like these three companies are building 80% of what's undone in getting publishers back their negotiating leverage in a data-driven world.

2. The only other pieces of the web-based ROI feedback loop that I articulated in February that remain unaddressed are the piece at the marketers and the piece at the individuals.  I am still actively looking for companies in the former.  I've met a couple of really exciting ones and am working with them to get to launch-readiness, but want to meet more; this is a big area.  I'm trying to figure out what, if anything, could work at the latter.  I would have invested in Hunch, had I the opportunity back then, but I think there are other ways to address the consumer side of the people-product matching problem.

3.  Social may be one.  The social loop will share superficial characteristics with the display loop, but it's really completely different.  A softer, more subjective approach is needed. IMHO, the area with the most near-term leverage will be tools that help communicators understand the impact of how they are communicating and then help them make better decisions.  I've made an investment here that will probably also be announced anon.  I'm treading carefully in other areas of social because I've seen a lot of ideas imported from the display ad world rise and plateau: social is different and harder to scale.

4. Mobile.  In addition to my long-ago investment in Pinch Media (now Flurry) I've made an investment in a geo-data startup.  This one has not announced publicly, but it's a big idea.  Other than that, again I'm treading carefully.  Steve Jobs' reaction to Flurry earlier this year is an example of why: no matter where you invest in mobile, you're at the mercy of some pretty ruthless gatekeepers.  That said, it's exciting to think about how much value startups could add through the mobile platform.

5. Data is a big problem.  And, to be clear, it's been a big problem for a long time so it has some big and expensive solutions sold by big and slow corporations.  But the companies that are bringing data-munging solutions into the reach of smaller companies operating without teams of specialists are pretty interesting.  I've been trying to get a feel for the process, on a small scale, as my last couple of posts show.  But I've got a lot to learn before I even begin to feel competent.  (If you have something to teach me, I'll buy lunch.)  I want to find companies that can democratize data science.

6. Respect for the individual.  I think about this a lot.  I don't talk about it a lot.  It's a difficult area and using a rational thought process on something that's pre-rational in our makeup is tough.  I haven't been able to organize my thoughts, so I have no idea how I support the value-creation process.  But it's always there in the back of my mind.  I am open to suggestions.

So that's what I've been doing and thinking.  Points 2 through 5 will probably continue to be my focus through most of 2011.

*****

In terms of the AdExchanger bit, here's what I was getting at.

At IBM in 1988, in meetings to discuss moving an instruction's execution from the microcode engine to the hard-coded execution engine, I realized that our design team was at the tail-end of a long chain of product-market fit interactions. Because of the way end-users were using applications, applications had changed which parts of the operating system were critical paths, and the operating system designers had come back to us hardware folk asking that certain instructions be optimized for performance.

This was part of a long-standing and ongoing co-evolution between the hardware and the software.  Innovations on the hardware side changed what software was viable.  Innovations on the software side changed what was needed of the hardware.  Read Melinda Varian's intensely interesting VM and the VM Community, where she talks about the development of IBM's time sharing operating system, and think about it as a co-evolutionary process.  IBM in 1964 opened an office in Cambridge as a liaison to the MIT software engineers developing the first time-sharing operating systems.  Because of its proximity to the users, this office was instrumental in pushing IBM to change its mindset from building machines optimized for batch processing to building machines optimized for time-sharing.

During the next few years, both sides--the operating system writers and the hardware designers--pushed the other side to optimize around what they thought was needed or what they could deliver.  The software engineers pushed for address relocation capability and other features needed for time-sharing. The hardware engineers spec-ed a new processor to meet those requirements.  The hardware engineers floated the idea of virtual machines.  The software engineers built an operating system to take advantage of them**. 

Calling this co-evolutionary may be oversimplifying.  In fact, each layer of the stack co-evolved with the adjacent layers.  Sometimes the tension between different layers evolving differently led to entirely new organisms forking off, like Multics (which then evolved into Unix and its descendants.)  This process can happen in any multi-layer ecosystem that has different actors in different layers.

The data-driven display ecosystem is like that.  The tensions between co-evolving layers is evident (AppNexus/Google anyone?)  And forks are emerging.  I expect that within three years there will be a major fork away from the owned exchanges into a crossing platform that can better support the demands of the adjacent layers: the data exchanges, analytics, the DSPs and the SSPs.  Right now none of these are especially happy with what the exchanges are offering.  Not to say that they're unhappy, but they each have a laundry list of things they would improve or change.  The best of them are creating ways to avoid the exchanges, but only because there is no good alternative.  A platform would still be most efficient.

The exchanges, on the other hand, seem to be pretty content with the way things are.  Or, at least, they feel that they should be controlling the pace and direction of the evolution.  This opens the way for entrepreneurs to build something disruptive.  If you're an entrepreneur with the chops to build something here, you should start thinking about it soon.  2012 will be too late.

-----
* No Thanksgiving jokes, please.
** One great anecdote talks about how the early CP/CMS OS could only keep one virtual machine in memory at a time.  So when a user logged on, the OS would reserve space on the paging drum for the copy of their virtual machine while somebody else's was running.  When the paging drum was full, space would be reserved on disk.  Since the paging drum was so much faster than disk, people started showing up to work earlier and earlier so they could get a slot on the drum.  Finally the OS designers made the page slot allocation mechanisms dynamic.  They just weren't morning people.