Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
| Date | Stars |
|---|---|
| 2026-07-24 | 2498 |
| 2026-07-25 | 2499 |
| 2026-07-28 | 2499 |
| 2026-07-30 | 2499 |
| 2026-07-31 | 2507 |
| 2026-08-06 | 2509 |
Today
+2 stars today
This week
+10 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.40%/day
## Hivemind: decentralized deep learning in PyTorch [](https://learning-at-home.readthedocs.io/en/latest/?badge=latest) [](https://pypi.org/project/hivemind/) [](https://discord.gg/uGugx9zYvN) [](https://github.com/learning-at-home/hivemind/actions)  [](https://github.com/psf/black) Hivemind is a PyTorch library for decentralized deep learning across the Internet. Its intended usage is training one large model on hundreds of computers from different universities, companies, and volunteers.  ## Key Features * Distributed training without a master node: Distributed Hash Table allows connecting computers in a decentralized network. * Fault-tolerant backpropagation: forward and backward passes succeed even if some nodes are unresponsive or take too long to respond. * Decentralized parameter averaging: iteratively aggregate updates from multiple workers without the need to synchronize across the entire network ([paper](https://arxiv.org/abs/2103.03239)). * Train neural networks of arbitrary size: parts of their layers are distributed across the participants with the Decentralized Mixture-of-Experts ([paper](https://arxiv.org/abs/2002.04013)). To learn more about the ideas behind this library, see the [full list](#citation) of our papers below. ## Example Use Cases This section lists projects that leverage hivemind for decentralized training. If you have successfully trained a model or created a downstream repository with the help of our library, feel free to submit a pull request that adds your project to this list. * **Petals** ([webpage](https://petals.dev), [code](https://github.com/bigscience-workshop/petals)) — a decentralized platform for inference and fine-tuning of 100B+ language models. * **Training Transformers Together** ([webpage](https://training-transformers-together.github.io/), [code](https://github.com/learning-at-home/dalle-hivemind)) — a NeurIPS 2021 demonstration that trained a collaborative text-to-image Transformer model. * **CALM** ([webpage](https://huggingface.co/CALM), [code](https://github.com/NCAI-Research/CALM)) — a masked language model trained on a combination of Arabic datasets. * **sahajBERT** ([blog post](https://huggingface.co/blog/collaborative-training), [code](https://github.com/tanmoyio/sahajbert)) — a collaboratively pretrained ALBERT-xlarge for the Bengali language. * **PyTorch Lightning Integration** ([docs](https://lightning.ai/docs/pytorch/stable/integrations/strategies/Hivemind.html)). Integration into PyTorch Lightning allows adapting your existing pipelines to training over slow network with unreliable peers. ## Installation Before installing, make sure that your environment has Python 3.8+ and [PyTorch](https://pytorch.org/get-started/locally/#start-locally) 1.9.0 or newer. They can be installed either natively or with [Anaconda](https://www.anaconda.com/products/individual). You can get [the latest release](https://pypi.org/project/hivemind) with pip or build hivemind from source. ### With pip If your versions of Python and PyTorch match the requirements, you can install hivemind from pip: ``` pip install hivemind ``` Also, if you want to use blockwise 8-bit compression from [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) during data transfer, you can install it with `pip install hivemind[bitsandbytes]`. After that, you can use the `BlockwiseQuantization` class in [hivemind.compression](./hive
Excerpt of 13,587 characters
Read on GitHub
justheuristic · YSDA
320
Max Ryabinin
95
Alexander Borzunov
81
Michael Diskin · Wildberries / HSE University / YSDA
17
Anton Gusev · @ispras · Russia
16
Denis Mazur · Netherlands
12
10
Alexey Bukhtiyarov · NLP Team Lead at Ex-Human
7
Pavel Samygin · WunderFund
3
Artem Chumachenko
3
Ryan Brooks
3
Anton Sinitsin · @Yandex
3
3
Egiazarian Vage · ISTA
2
2
Roman Zhytar
2
2
Dmitry Baranchuk · Yandex Research
2
1
samsja · @PrimeIntellect-ai · United States
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:8568ed27a5268aef, topic:deep-learning, topic:pytorch, topic:distributed-training