Running large language models across a peer-to-peer cluster of edge devices
HyperCluster is my capstone project: a fully decentralized framework for running LLM inference across a cluster of ordinary edge devices connected over a wireless network, with no coordinating server.
The memory footprint of a modern LLM is the obvious problem — no single phone or laptop in the cluster can hold the whole model. HyperCluster addresses this with two pieces:
Ring-based pipelined inference. Nodes deterministically self-organize into a computational ring ordered by device capability. Each node runs its shard and passes intermediate hidden states directly to its successor, so activations never round-trip through a central coordinator.
Generalizable model sharding. Built on top of HuggingFace Transformers, the sharding layer automatically partitions any dense LLM across heterogeneous devices according to the compute and memory each one actually has available — rather than assuming a uniform cluster.
Peer discovery and P2P transport are handled with Iroh, so nodes find each other and form the ring without manual configuration.
The work draws on ideas from exo for sharding and prima.cpp for ring pipeline architectures.
HyperCluster: Decentralized Large Language Model Inference over Peer-to-Peer Wireless Networks
P. Samarth, Vyoman Jain, Sanjiv Raghunandan, and 2 more authors
In IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS). Also presented as a poster at the ML4Wireless workshop, AAAI 2026 (non-archival), 2026
Large language models’ substantial memory and compute requirements hinder deployment on individual resource-constrained devices. HyperCluster is a framework for fully decentralized collaborative inference over wireless networks, contributing a ring-based pipelined inference protocol in which nodes deterministically self-organize into a computational ring by device capability and pass intermediate states directly between peers, and a generalizable model sharding methodology built on HuggingFace Transformers that automatically partitions any dense LLM across heterogeneous devices according to available compute and memory.
@inproceedings{samarth2026hypercluster,title={HyperCluster: Decentralized Large Language Model Inference over Peer-to-Peer Wireless Networks},author={Samarth, P. and Jain, Vyoman and Raghunandan, Sanjiv and Sri, Akepati Ramya and Sharma, Richa},booktitle={IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS)},year={2026},publisher={Springer},doi={10.1007/978-3-032-27358-1_2},}