Platforms and Adaptive Systems

Whether in business or biology, complex systems repeatedly exhibit platform-application dynamics, with separation between broadly useful, hard-to-implement components in the platform, and highly adaptive, easy-to-implement systems on top. Why is this the case? It’s bad for every use case to reinvent the wheel, and it’s also bad if a slow development process prevents the discovery of new niches. This post explores this dynamic in both nature and technology.

Nature

Why are humans so successful? Yes, we have more compute power and tools, but the real key is that humans are adaptable. Our flexible intelligence allows us to constantly model, predict, and optimize our approach to meeting our goals.

To illustrate, consider an ant. It exhibits intelligent behaviors: it can walk, it can find and harvest food, and it can reproduce. But these behaviors are baked into their genetic code, and they can’t just decide to mix it up. Ant behaviors can change over time; at one point leaf cutter ants “realized” they can consume leaves. But look at the name - it’s a different species of ant that has this behavior, and new species take a lot of time to form. Random evolution might have caused their ancestors to climb trees, then accidentally eat leaves while consuming caterpillars, then they continued to eat leaves after the caterpillars were gone, and they ultimately survived as a result. No single ant decided, “Hey boss, let’s try eating those abundant green things up there”.

An ant, failing to realize that it could eat this leaf and provide the colony with a new, abundant food source. Photo by mi_shots on Unsplash.

Humans appeared on this planet only very recently, and by all measures have made a splash. How have we completely dominated nature and every other species on the planet? This reconfigurability leads to faster iterations. Evolution is a wonderful engineering paradigm; it leverages massively parallel compute (across all instances of creatures on the planet), and given enough time it can produce spectacular results. But it’s slow, while the development and propagation of human adaptations is fast.

Woolly mammoths took millennia to grow thick coats of fur. Give a human a knife and a dead animal on a cold day, and they’ll achieve the same in an afternoon. Even faster if their parents told them how to do it. Just by living, each human automatically writes a new, highly customized app on the human/brain platform. Eat your heart out, bubble.io.

Higher adaptability in a dynamic world leads to a higher rate of improvement. Due to compound interest, “the most powerful force in the universe”, these gains add up. Shown is a toy model with a 1/1000th vs. 1/10th of a percent yearly improvement over 10000 years.

On the other hand, humans don’t appear as amorphous blobs of matter, with every biological system engineered from scratch at birth. While someday our reconfigurability will extend to our hardware, right now a lot is baked in from the start: our physical bodies, our love of calorie-rich foods, and our desire to reproduce, among many others. A creature that had to re-develop the entire stack would be at a disadvantage, since there are some “platform capabilities” that are (almost) always useful. Why reinvent how we extract energy from glucose every generation, since it’s costly to do so and unlikely to work? See Behave by Sapolsky for more.

Each human automatically builds a new app (their personality and capabilities) on the human/brain platform that is highly adapted to their environment.

This balance needs to be finely tuned. Reconfigurability comes at a cost, but confers an advantage whenever we’re in a dynamic, changing environment, and there’s a long tail of use cases to be addressed.

Platform Principles (iOS)

Now let’s turn our attention to a famous platform: the app store. the iPhone and iOS provide a set of core capabilities, with high general utility and low reconfigurability. People will interact via touch, there is one place for notifications, and even UI elements generally look the same. App developers build on top of this to solve a wide array of more specific problems, iterating rapidly towards their goal of finding something that will generate a profit. This is clearly more productive than each app reimplementing everything down to the silicon.

Let’s be a bit more specific. An enterprise will tackle a problem if the benefits outweigh the costs. If a capability is useful across many apps, the aggregate value can be massive; a platform can thus justify significant cost to realize this value. If a capability is only useful for a specific use case, the platform will leave it to the app developer, if they can find a way to achieve positive ROI. Of course, the platform developer has an incentive to make implementation of any given use case as easy as possible, but there will always be some additional work.

Toy model showing the user value of each capability for each scenario, and the sum across scenarios.

Why doesn’t the platform developer just create all of the apps themselves? Central planning can only take us so far! It takes an enormous amount of information processing (for humans) to solve problems for a long tail of customers and use cases. Apple can align thousands of engineers to solve a particular set of problems (make a useful, beautiful, general purpose computing device), but no single company can excel at discovering and solving thousands of different business objectives at once. Each enterprise focuses on their own specific set of problems.

The tricky part is that the platform developer usually doesn’t know a priori how it’s going to be used. It’s dangerous to guess. It’s not obvious that a glass rectangle that senses touch and has great developer libraries will produce valuable applications. Plus, you need users to attract developers. It makes sense, then, that Apple built some of the apps themselves, to get users, prove the value, and harden and improve the platform capabilities before opening up the easier-implemented parts of the stack for third parties. When done right, the integrated solution has the right platform abstractions built in from the start, and these abstractions and tools make development as easy as possible for these third parties in the future.

Remember that the same thing occurred in Nature! The organism first evolved to survive by making something work end to end in a non-flexible way (everything driven by genes) before generalizing some highly adaptable infrastructure (your brain). It’s also worth making explicit that this is more of a continuum than a dichotomy. Some parts of our biology have been around since the dawn of life (e.g. the ribosome), and others are more ephemeral and arise based on the use case (e.g. Jellyfish acquire and discard eyesight quite often). In science and engineering, we tend to keep the laws of physics fixed and iterate more as we move up the stack.

Examples

At the risk of belaboring the point, it’s interesting seeing how frequently this dynamic plays out with some more examples.

Amazon Web Services

A classic example is Amazon’s highly successful cloud business. They first figured out what cloud technologies are needed to run a global web service by running Amazon.com, and then opened up some of the core components for third parties for them to apply to their own use cases. They enforced that internal teams had to interact via APIs to ensure they would build hardened interfaces that could subsequently be turned on for outside developers; nailing this interface is nontrivial, and key for the success of the platform.

Amazon Marketplace

Another less-cited example is Amazon Marketplace. They solved the problem of customers wanting selection, low prices, and fast shipping by integrating and going end-to-end, and then opening up their website and fulfillment network (big, costly components to produce) for third parties to sell their own products. They would compete with each other, and with Amazon itself, to provide the best price and shipping speed within their respective categories, and this helped the platform as a whole sell more products at better prices than would have been possible if Amazon was doing it alone. Marketplace accounts for 20% of revenue, and is a massive driver of profit growth today.

Transfer Learning

Changing gears, a technique in machine learning exhibits the same properties. Let’s say you want to train a machine learning model to classify bird species. You could assemble a dataset with millions of examples of birds (obligatory xkcd link) and train a deep neural net, but it’s going to take a lot of money and time to do so. The model is learning from scratch to recognize shapes, lines, beaks, bird faces, etc. before it can even start to understand the difference between a finch and a parakeet.

With transfer learning, you can take a model that has already been trained on an existing, large, diverse dataset (e.g. ImageNet), and fine tune it for a new, related task. Some aspects of pattern recognition are broadly useful, and expensive to produce; these are the reusable “platform” capabilities. Building, or in this case, learning, on top of these tools allows you to quickly adapt the model for birds, cars, or anything you might want, at a much lower cost.

Computing Machinery

Before Turing came along, early computers were purpose-built for computing artillery trajectories. One of the great intellectual achievements of the modern age is that this was generalized, not just to a few other specific applications, but to anything that is “computable”. Instead of a new application requiring a physical circuit to implement its logic, one company can focus on making really good general computing hardware, and anyone can come along and provide instructions to run on top.

Immune Systems

The human immune system has the general ability to identify, clear, and remember clear pathogens (platform), and can adapt to recognize new pathogens as they’re discovered (app). Similar story for bacterial immune systems as well (i.e. CRISPR).

Closing Thoughts

One might say that platform-app divisions are inevitable in complex systems in a dynamic world, whenever there are pieces that are broadly valuable and hard to build, and others that are specific to niches and easy to build. Keep an eye out in code, organizations, society, nature, and beyond. Comment below with other examples!

Many strive to build the next great platform. Hopefully this post can help you think about what scenarios will benefit from a new platform, what should be baked into the platform vs. left to the app, and how to discover and launch with the right abstractions in place.