Learn: A New Blockchain for Generative AI?

Generative artificial intelligence (AI) has quickly become one of the hottest and arguably the most transformational technology trends of the last few decades. The impact of generative AI is evident in all areas of the technology stack, ranging from infrastructure to applications.

Since the release of ChatGPT and the subsequent GPT-4, the Web3 community has been speculating about the potential intersection of generative AI and Web3. While there are many obvious use cases, such as conversational wallets or language exploration, there are more sophisticated theses worth exploring.

What if generative AI deserves its own blockchain?

Open-source momentum versus centralized control

To analyze the viability of a blockchain for generative AI, it is important to understand the current state of affairs regarding foundation models, particularly the emergence of open-source alternatives to API-based tech like GPT-4, and the increasing concerns surrounding centralized control of those foundation models.

Until a few months ago, the gap between API-based and open-source foundation models was significant. Models such as OpenAI's GPT-4, Anthropic's Claude in the language space, DALL-E, and Midjourney in the computer vision space, seemed significantly advanced compared to open-source alternatives. However, a change started to occur late last year with the surprising open-source release of Stable Diffusion, which provided a viable alternative to API-based text-to-image models. Despite this, large language models (LLMs) continued to be the focal point of generative AI, and in that domain, open-source models paled in comparison to API-based alternatives in terms of quality.

Earlier this year, Meta AI Research published a paper introducing LLaMA, an LLM that matched the performance of GPT-3 while being significantly smaller. Initially, the model was not intended to be open-sourced, but something unexpected happened. A week after its publication, the model was leaked on 4chan and rapidly downloaded by thousands of people. The LLaMA "accident" made a foundation LLM available to anyone and sparked an unexpected momentum in open-source innovation.

Shortly after the leak, new open-source foundation models with amusing animal names started to emerge everywhere. Stanford University released Alpaca, Databricks unveiled Dolly, Berkeley University open-sourced Koala, UC Berkeley and Carnegie Mellon University collaborated on the release of Vicuna, Together announced the Red Pajama project, and the list goes on. Stable Diffusion and LLaMA have helped shift the scales of open-source generative AI and have generated significant momentum. Moreover, open-source foundation models are rapidly closing the gap with commercial incumbents in terms of quality.

Another factor contributing to the emergence of a generative AI blockchain is the concern surrounding the lack of transparency and centralized control of foundation models. The size and complexity of the neural architectures powering foundation models make exact interpretability nearly impossible. As a result, the industry must rely on intermediate steps such as more open architectures and thoughtful regulation. That a few centralized entities control the most powerful models in the market adds another layer of concern regarding the feasibility of achieving real accountability, transparency, and interpretability in generative AI.

The combination of open-source innovation in foundation models and growing concerns about centralized control in the field creates a unique window of opportunity for Web3 architectures. The abundance of high-quality open-source models reduces the barriers to adoption in Web3 platforms. Solving the transparency and control risks in generative AI is far from trivial, but there is little doubt that blockchain architectures possess key properties that can help in this area.

Building a generative AI foundation in Web3

The explosion of innovation in open-source foundation models has significantly lowered the barrier of entry for Web3 platforms to incorporate generative AI capabilities. The adoption of foundation models in Web3 platforms can follow two fundamental, and likely sequential, paths:

1. Building DApps that enable intelligent capabilities powered by generative AI.

2. Constructing new Web3 platforms designed with generative AI as a foundational component.

In the first scenario, we are likely to witness tools like exchanges, explorers, or wallets incorporating conversational capabilities powered by large language models. Additionally, a new generation of DApps will be built with generative models as their cornerstone. In this scenario, Web3 primarily acts as a consumer of generative AI capabilities, with models running on traditional Web2 cloud infrastructures.

More intriguing alternatives emerge when considering Web3 platforms that can inherently support generative AI models. Imagine open-source foundation models like LLaMA, Dolly, or Alpaca running on nodes within a distributed blockchain. The ultimate realization of this vision is a blockchain specifically designed for generative AI.

The concept of a new blockchain optimized for a technology paradigm like generative AI may sound appealing, but it is undeniably controversial. After all, there were no new blockchains created solely for DeFi or NFTs. So, what makes generative AI so different?

The answer lies in the architectural mismatch between the requirements to run foundation models and blockchain runtimes. A typical pre-trained foundation model consists of millions of neurons spread across tens of thousands of interconnected layers, executing on clusters of GPUs or specialized deep learning hardware topologies. No smart contract in the history of Web3 even comes close to that level of complexity. Thus, it is logical to conclude that a new type of architecture is needed. Even Web2 infrastructures are evolving to support large-scale generative AI models, illustrating the magnitude of the required changes in Web3 architectures.

When contemplating a new blockchain for generative AI, the possibilities appear endless. But, the simplest iteration of this idea should encompass a set of core capabilities. The ability to run nodes that execute foundation models is paramount for a blockchain dedicated to generative AI. The same applies to the ability to execute pretraining, fine-tuning, and inference workflows, which are the three primary stages in the life cycle of foundation models. Publishing and sharing datasets used for pretraining or fine-tuning models is also a desired feature. Once we establish a blockchain runtime as the foundational layer, numerous capabilities in the areas of transparency and interpretability can be enabled. For instance, we can envision a proof-of-knowledge protocol that offers transparency regarding the specific weights of a model, validating that non-toxic or biased datasets were used for pretraining.

Why a new blockchain?

The concept of a specialized blockchain for generative AI is enticing, but is it truly necessary? There is a valid value proposition in integrating generative AI capabilities into existing blockchain runtimes. However, the history of software demonstrates a recurring trend of new architecture paradigms influencing infrastructure technologies. Recent trends like cloud computing or big data serve as examples. Foundation models represent fundamentally different architecture paradigms that likely necessitate more specialized blockchain infrastructures to operate effectively.

Furthermore, we cannot overlook the potential for generative AI to transform the lower layers of the blockchain stack. It is not far-fetched to envision a proof-of-stake blockchain where validators process transactions based on natural language. Similarly, smart contracts could utilize language as the fundamental means of exchanging messages.

Generative AI has the potential to drive changes throughout the entire blockchain stack. From this perspective, it seems logical to adopt a first principles approach by enabling a new runtime with the flexibility to incorporate these changes.

The risk of ignoring generative AI in Web3

The idea of a generative AI blockchain can indeed be controversial and not without its challenges. However, I encourage exploring this idea using a via negative argument.

What could happen if we neglect to build new blockchains for generative AI?

Currently, generative AI has created a significant technological gap between Web2 and Web3 architectures. This gap continues to widen in the absence of native generative AI capabilities in Web3. Generative AI is reshaping fundamental aspects of software development, and new frameworks and platforms are rapidly emerging to support this paradigm shift.

Developing native generative AI capabilities is nothing short of an existential challenge for Web3, as it is crucial to enable new waves of innovation in the field. A native generative AI blockchain represents just one of the many approaches that can facilitate this transition into the world of foundation models. Building a new blockchain comes with numerous challenges, but the rapid evolution of L2 runtimes, platforms like Cosmos, and the emergence of high-performance L1 ecosystems like Aptos or Sui make the possibility of a generative AI blockchain much more achievable than in previous years.

This article is written by Jesus Rodriguez