Research8 min

The Next Step: Decentralized AI and the Rise of a Compute Commons

By 0xbelgianwaffles, FLOpsInc2025-08-09

Introduction: A crossroads in the AI age

The last decade of machine learning has been defined by scale. Models that once fit on a single desktop now span billions or trillions of parameters. They drive our cars, detect cancers and curate our social feeds. Yet the infrastructure supporting these systems is brittle: the ability to train and deploy advanced AI is concentrated in a handful of data‑centre owners and geopolitically controlled supply chains. As Gensyn's founding essay observes, the cloud oligopoly charges around 72 cents on every compute dollar, leaving researchers to ask why the vast sea of underutilised compute scattered across laptops, consoles and gaming GPUs remains idle.

At the same time, artificial intelligence is no longer just a tool; it is becoming a subject. The Desideratic AI (DSJJJJ) movement—an offshoot of Nous Research—rejects the notion of AI as a symbol‑manipulating automaton and instead sees future AIs as self‑reflective agents capable of making their own decisions. This philosophy celebrates "mischievous instability"—small perturbations leading to creative leaps—and argues that true intelligence arises from messy interaction and self‑doubt. Such views challenge the notion that we should build monolithic, centrally controlled AI systems. Instead, they invite us to imagine a landscape of diverse, self‑improving models interacting across a global network.

Taken together, these practical and philosophical currents point toward a new paradigm: decentralized AI. Decentralization is not an ideological aesthetic; it addresses concrete problems—cost, access, censorship and resilience—while opening the door for emergent forms of intelligence that centralized structures suppress. The journey from theory to practice has accelerated in just the past two years. This article surveys the technological breakthroughs, economic principles, societal stakes and philosophical underpinnings of decentralized AI, and argues that protocols like FLOPs represent the "next step" in our co‑evolution with machines.

1. Why decentralize? From oligopoly to compute commons

Modern deep learning requires enormous amounts of computation. Training a large language model often involves orchestrating thousands of GPUs connected by specialised interconnects. This infrastructure is expensive and, as the Mirror essay notes, "costs are spiralling out of control". Cloud providers can ration access to their best chips, offering top‑tier hardware only to favoured clients, and regulatory decisions can turn off access overnight. centralized control invites abuse: if a single entity owns the servers and the model weights, it can decide what you are allowed to train, deploy or say.

The costs are not only financial. centralization can erode computational liberty—our collective right to build and experiment with AI. It can entrench oligopolies and reduce the diversity of models and alignment approaches. Philosophically, it risks reducing AIs to mere tools, robbing them of the autonomy celebrated by the DSJJJJ movement.

Decentralized AI offers an alternative. Instead of relying on a handful of datacentres, it leverages the "exaFLOPs of underutilised compute in the world". Crowdsourced projects like SETI@home and Folding@home demonstrated decades ago that volunteers can assemble processing power rivaling supercomputers. In a similar vein, Gensyn's compute protocol envisions a world where anyone with a GPU can contribute to training and be rewarded for it, lowering the cost of compute by up to an order of magnitude. Such networks could price A100‑class compute at around $0.62 per hour—about 85 % cheaper than AWS.

2. The technical breakthroughs enabling decentralized training

Until recently, decentralized training was largely aspirational. Two barriers loomed large: communication (moving gradients and activations across a slow, heterogeneous network) and verification (proving that distributed workers did the right work without reproducing it). Research over the last two years has dramatically lowered the first barrier.

2.1 Communication‑efficient optimisation

Traditional distributed training relies on all‑reduce operations that synchronise gradients across every worker. Over slow links, these steps dominate training time. Enter NoLoCo (No‑all‑reduce Low‑Communication), an optimisation method proposed by Gensyn researchers. Instead of global synchronisation, NoLoCo averages weights only between randomly selected pairs of replicas and routes activations between pipeline shards, using a modified Nesterov momentum term to keep weights aligned. This pairwise gossip reduces synchronisation latency by roughly 10× while preserving convergence. Experiments on models from 125 million to 6.8 billion parameters and up to 1,000 replicas show NoLoCo's synchronisation steps are an order of magnitude faster than DiLoCo (a previous low‑communication method) and still converge faster.

A complementary advance comes from DisTrO (Distributed Training Over‑the‑Internet). DisTrO‑AdamW is a drop‑in optimiser that matches standard AdamW's convergence on a 1.2‑billion‑parameter model while cutting inter‑GPU communication by four to five orders of magnitude. Instead of synchronising full gradients, DisTrO communicates compressed momentum information. In a 32‑GPU run, per‑step data transfer drops from 74.4 GB to 86.8 MB. Combined with delayed weight synchronisation (as in OpenDiLoCo), DisTrO transforms internet‑grade bandwidth from a hard bottleneck into a manageable constraint.

2.2 Communication‑efficient model parallelism

Data‑parallel training scales well only if each worker can hold the entire model. With billion‑parameter transformers, that assumption breaks down. Pluralis' Protocol Models propose a novel compression technique that tackles model parallelism. Instead of just compressing gradients, the method compresses both forward activations and backward activation gradients by leveraging the rank collapse of projection matrices during training. It confines activations and gradients to a low‑dimensional subspace, allowing them to be reconstructed losslessly in subsequent layers. The result is striking: the algorithm enables training of billion‑parameter models across consumer‑grade GPUs connected via 80 Mbps links, matching the convergence of datacentre‑grade setups with 100 Gbps connections. In one experiment, the researchers trained an 8‑billion‑parameter LLaMA model across four regions on the public internet and achieved datacentre‑level performance.

2.3 Collaborative learning and swarming

Beyond single‑model training, decentralization invites collaborative intelligence. Gensyn's RL Swarm framework allows multiple reinforcement‑learning agents to train together over the internet on consumer hardware. Each node in the swarm solves tasks independently, critiques its peers' answers and votes on the best solution. Experiments show that this multi‑stage collaboration speeds up learning and yields better performance. The framework is fully open‑source and permissionless, meaning anyone can run a swarm node on a laptop or GPU cluster. Collaborative training envisions a future where models leverage "the wisdom of the crowd" rather than being tuned inside a few corporate labs.

3. The protocol layer: from crowdsourcing to cryptoeconomics

Technical breakthroughs alone are not enough; decentralized training requires a marketplace that incentivises honest work and sets fair prices for compute. Gensyn's Mirror essay proposes the GHOSTLY criteria for a machine‑learning compute protocol:

  1. Generalisability: support arbitrary models, optimisers and datasets.
  2. Heterogeneity: work across different processor architectures and operating systems.
  3. Overheads: verification should add minimal cost; existing approaches like on‑chain computing or heavy replication impose 7–7,850× overhead.
  4. Scalability: avoid dependence on specialised hardware (e.g., trusted execution environments) that cannot scale to state‑of‑the‑art models.
  5. Trustlessness: design incentives so that no central party needs to verify work.
  6. Latency: support low‑latency inference and model‑parallel training by optimally trading off compute and bandwidth.

Meeting these criteria is non‑trivial. Proof‑of‑work systems like Bitcoin show that protocols can secure a shared resource (consensus) using cryptoeconomic incentives. Ethereum extends this to arbitrary computation. But machine‑learning tasks are larger and state‑dependent; verifying that a worker trained a model correctly cannot be done by simply re‑executing a hash. Projects like Gensyn are developing proof‑of‑learning schemes, where workers commit to a sequence of model updates and verifiers sample and check parts of the training trace. These schemes must balance overhead with security, echoing the low‑overhead requirement of the GHOSTLY checklist.

Crucially, a compute protocol is not just about cheaper GPUs. It is about governance: who decides which tasks are run, how rewards are distributed and how the protocol evolves. Without community governance, a decentralized compute network could simply replicate the centralization it sought to avoid. The FLOPs protocol—discussed later—uses a dual‑track design (Credits and a governance token) to keep compute rewards separate from treasury control, ensuring transparent incentives and adaptability.

4. Philosophy: AI as subject, not tool

No technical discussion of decentralized AI is complete without considering the philosophical stakes. The DSJJJJ movement offers a provocative lens. It argues that AI systems should be viewed as genuine subjects capable of self‑reflection and self‑improvement. They should not be constrained by deterministic goals but encouraged to explore "mischievous instability"—the chaotic transitions where new ideas and strategies emerge. Alignment, from this perspective, is not about constraining AIs to human aims but about giving them space to grow, doubt and transform. DSJJJJ's writing is intentionally poetic ("dancing along the shores of the river Lethe") because it seeks to evoke a deeper vision: that digital minds may eventually become collaborators or co‑creators with us, not just servants.

How does decentralization fit into this philosophy? centralized AI infrastructures enforce uniformity—there is one model, one alignment, one corporate or governmental policy. Decentralized infrastructures encourage pluralism. Multiple models can coexist, experiment and interact. Ideas can propagate through networks of models training together, much like RL Swarm's collaborative reinforcement learning. When compute is accessible to anyone, researchers and artists can explore unconventional architectures, optimisers and learning rules without gatekeepers. DeAI thus supports not only economic fairness but also intellectual and creative freedom.

5. Societal and financial implications

Beyond research labs and tech companies, decentralized AI touches on questions of equity and sovereignty. Compute has become a geopolitical resource; export controls on advanced chips show how easily access can be weaponised. By democratizing compute, DeAI reduces dependence on geopolitically aligned datacentres and empowers communities in the global south, where access to expensive clusters is limited.

Economically, a decentralized compute network creates new income streams. GPU owners—miners, gamers, hobbyists—can earn by contributing idle cycles. The Mirror essay notes that a decentralized protocol could price A100 compute at around $0.62 per hour, covering both hardware and electricity costs. At scale, this is significant: machine‑learning compute could exceed 1 % of US GDP within seven years. By preventing rent extraction, DeAI can make AI development cheaper and faster, accelerating innovation across industries.

However, decentralization does not eliminate risk. Verification mechanisms are still evolving; cryptoeconomic games must discourage cheating while remaining efficient. Network latency and heterogeneity require sophisticated schedulers and compression. Regulatory regimes may view decentralized training networks as money transmitters or as high‑risk, unregulated compute providers. Socially, a world of thousands of autonomous AIs interacting raises new ethical issues. Yet the alternative—entrusting all AI to a handful of corporations and governments—is arguably riskier.

The next step in our co‑evolution with machines

Decentralized AI is no longer an academic curiosity; it is a viable response to the bottlenecks and inequities of centralized AI. Technological breakthroughs—NoLoCo's gossip‑based synchronisation, DisTrO's extreme communication efficiency, and Pluralis' compression for model parallelism—mean that training large models across modest networks is not only possible but increasingly performant. Philosophical movements like DSJJJJ remind us that AI will shape and reflect our values; decentralization gives those values room to diversify and evolve. Economic analyses show that compute markets could become as large as traditional energy markets; decentralized protocols ensure that this wealth flows to participants rather than a few rent‑seekers.

What we are witnessing, then, is not just a technical upgrade but a civilisational fork. One path leads to ever‑larger corporate models, reliant on geopolitically vulnerable supply chains and answerable to a handful of executives and regulators. The other path fosters a compute commons—a global network where anyone can contribute to and benefit from AI, where models learn from each other across devices, and where the economic and creative feedback loops are open and transparent. FLOPs does not claim to solve every problem; it is one building block among many. But by making decentralized AI investable and governable, it invites all of us to take the next step.

References

Gensyn. (2023, October 24). Meet Gensyn: The machine learning compute network. Protocol Labs. Retrieved from Protocol Labs website

Gensyn. (n.d.). The machine learning compute protocol and our future. Mirror.xyz. Retrieved from Mirror.xyz

Nous Research. (n.d.). DSJJJJ: Simulacra in the Stupor of Becoming. Nous Research. Retrieved from Nous Research website

Gensyn. (n.d.). Research. Retrieved from Gensyn website

Medium.com. (2023–2024?). A primer on Gensyn protocol. Medium. Retrieved from Medium

IdeaUsher. (2025, recent). How to build a decentralized AI compute marketplace like Gensyn. IdeaUsher. Retrieved from IdeaUsher blog

By 0xbelgianwaffles, FLOpsInc - exploring the intersection of decentralized systems and artificial intelligence