[92] AI Compounds Through Open Source
[AI Open Source Movement, Compounding in AI, Composability]
GM Readers!
Sharing an essay from our Evolving Internet Insights Newsletter unpacking how AI compounds through open source.
-LJW
(P.S. For the summary and further breakdown of this essay, follow me on Twitter)
Note: Some sections of this essay were originally released on my Evolving Internet Insights newsletter.
One of our reader friends and leading professors at Harvard Business School (HBS) has a fun riddle he asks his students to illustrate the power of compounding. It goes something like this.
Imagine you can fold a piece of paper in half as many times as you need to. How many folds will it take for it to reach the moon? 🌕
The most common answer he gets to this question is “a lot of folds.” While the common response is technically immeasurable, student’s minds pretzel trying to imagine the amount of folds it would take for this paper to reach the moon.
Surprising to many, the answer is 42 — by folding a regular piece of paper (0.1mm thickness) in half 42 times, it would reach the moon. 🤯🤔
Here is the math: (0.1mm paper thickness) x (2^42) = 439,805 km worth of folded paper and the average distance from Earth to the Moon is 384,400 km.
Intuitively, some of us understand that this is a “doubling function,” meaning that every time you fold the paper in half, you are doubling the thickness. We know it’s compounding.
What surprises most people is just how powerful compounding actually is.
Our friend’s point is that “humans are terrible at internalizing the effects of compounding.”
We love sharing story because it highlights why emerging technology adoption feels slow to start, but then “all of a sudden,” the technology feels ubiquitous, touching many aspects of people’s lives.
The proliferation of the Internet is an excellent example of the power of compounding. Since the widely-recognized “birth of the Internet” on January 1, 1983, nearly 5 billion people are now Internet users. This feat took 41 years to achieve – not quite 42 😜.
We believe the proliferation of AI will similarly benefit from compounding.
The Open Source Movement in AI
A while back, there was a popular essay, “Google ‘We Have No Moat, And Neither Does OpenAI”, that was released based on a leaked memo from Google, highlighting that the company has no moats concerning AI. The main argument centers on the proliferation of open source AI models that companies and developers can use without having to pay to access the proprietary models built by the Big Tech giants. Ultimately and over time, the cost and difficulty of using AI will decrease, reinforcing the main argument in the essay above – that there is no moat (defensibility) for Big Tech companies concerning AI.
The essay does a good job of highlighting an emerging trend in the AI space: the rise of open source models and platforms. This trend is not just being driven by startups and enthusiast developers on the Internet, but big companies like Meta are also leading the charge in open source.
Meta launched its open source large language model (LLM), Llama 2 for developers. Llama 2 is free to download, modify, and deploy, however Meta prohibits using Llama 2 to train other language models. Meta’s President of Global Affairs, Nick Clegg’s take on the benefit of open source models:
“With the … wisdom of crowds you actually make these systems safer and better and, crucially, you take them out of the … clammy hands of the big tech companies which currently are the only companies that have either the computing power or the vast reservoirs of data to build these models in the first place.”
On the scaleup front, Hugging Face, the leading AI and machine learning tooling hub for developers, has built a GitHub-like ecosystem for AI libraries, models, datasets, and code repositories. Hugging Face recently raised $235M in venture funding from investors that include: Google, Amazon, Nvidia, Intel, and AMD. This most recent round of funding values Hugging Face at $4.5B. At the time of funding, Hugging Face noted that they had 10,000 customers, 50,000 organizations on the Hugging Face platform, and over over 1 million code repositories.
Similarly, Stable Diffusion, one of the leading generative AI image generation models also open sources their technology and have created a fledgling community that contributes to and improves upon the base models by building applications on top.
The open source movement in AI extends far beyond AI models like LLMs and image generation systems. It has become a philosophical stance, with many believing that a technology as fundamental and powerful as AI needs to be open and accessible for others to use.
The Advantages of Open Source
One of the most compelling advantages of the open source approach is the acceleration of innovation through a concept called composability.
Composability refers to the ability to combine functionality from different pieces of code to create new and generally more complex functionality. In a composable system, different components can be selected and assembled in various combinations to achieve a specific goal. With composability in mind, when a developer creates something and open sources it for others to use, a new developer doesn’t have to start from scratch. The new developer can write their own code to also build on top of what was leveraged as open source.
Because AI models and repositories are open source, every time a problem is solved, it becomes a “building block” that others can use to build more complex products and applications. This results in a virtuous cycle where the sum is greater than its parts, enabling rapid technological advancement and reducing redundant or wasted efforts. For example, a developer can log onto Hugging Face and find an open source LLM that was built by someone else that is optimized for a specific use case, modify it with their own code, and deploy the new and improved LLM into their product.
Create the foundation once, and allow everyone to use and improve it.
But composability doesn’t just apply to writing software code, the concept has transcended into the culture. Those who identify with this concept typically have an open culture of experimentation and “building in public” that allows everyone to learn from the successes and failures of these experiments.
This all decreases the costs of innovation and experimentation, enabling more developers to build more software “building blocks” that others can use.
The Power of Compounding
In an open source ecosystem, someone jumping into AI for the first time benefits from all the work of those that came before them.
It is about standing on the shoulders of “giants.”
Bringing this back full circle: our reader friend often follows up his first question by asking his HBS class with, “if it takes 42 folds to reach the moon, how many more folds does it take to get back to Earth (from the Moon)?”
It only takes one (1) more fold to get back to Earth from the Moon. 🌝📜🌎
The 42nd fold gets you to the moon. The 43rd fold gets you back to Earth.
That is the power of compounding — and we see this everywhere in the AI space.
🚀🌕
Additional Readings: Here, Here, Here, Here
SHAMELESS Ask:
🙏 Share Life in Color with a friend and ask them to subscribe
🙏 Share on Twitter / Linkedin with a short note
🙏 Share on your company Slack / Teams channels and communities
🛟 Disclaimer:
This post is provided for educational and informational purposes only. Nothing written in this post should be taken as financial advice or advice of any kind. The author(s) may own some of the NFTs, art and/or collectibles mentioned in this post. The content of this post are the opinions of the authors and not representative of other parties.
Empower yourself, DYOR (do your own research).
 | A guest post by
|