Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Build high-performance AI models with modular building blocks
| Date | Stars |
|---|---|
| 2026-07-24 | 597 |
| 2026-07-25 | 598 |
| 2026-07-28 | 598 |
| 2026-07-30 | 598 |
| 2026-08-08 | 599 |
| 2026-08-16 | 599 |
| 2026-08-27 | 600 |
| 2026-09-08 | 601 |
| 2026-09-18 | 603 |
| 2026-09-20 | 603 |
Today
— stars today
This week
+2 stars this week
This month
+4 stars this month
Momentum
2.0
growth rate 0.33%/day
<p>
<a href="https://github.com/kyegomez/zeta/blob/main/LICENSE">
<img alt="MIT License" src="https://img.shields.io/badge/license-MIT-blue.svg" />
</a>
<a href="https://pypi.org/project/zetascale">
<img alt="PyPI" src="https://badge.fury.io/py/zetascale.svg" />
</a>
<a href="https://zeta.readthedocs.io">
<img alt="Docs" src="https://readthedocs.org/projects/zeta/badge/" />
</a>
</p>
**Zeta** is a modular PyTorch framework designed to simplify the development of AI models by providing reusable, high-performance building blocks. Think of it as a collection of LEGO blocks for AI each component is carefully crafted, tested, and optimized, allowing you to quickly assemble state-of-the-art models without reinventing the wheel.
<p>
<a href="https://discord.gg/EamjgSaEQf">
<img src="https://img.shields.io/badge/Discord-Join%20our%20server-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join our Discord" />
</a>
<a href="https://www.youtube.com/@kyegomez3242">
<img src="https://img.shields.io/badge/YouTube-Subscribe-red?style=for-the-badge&logo=youtube&logoColor=white" alt="Subscribe on YouTube" />
</a>
<a href="https://www.linkedin.com/in/kye-g-38759a207/">
<img src="https://img.shields.io/badge/LinkedIn-Connect-blue?style=for-the-badge&logo=linkedin&logoColor=white" alt="Connect on LinkedIn" />
</a>
<a href="https://x.com/kyegomezb">
<img src="https://img.shields.io/badge/X.com-Follow-1DA1F2?style=for-the-badge&logo=x&logoColor=white" alt="Follow on X.com" />
</a>
</p>
## Overview
Zeta provides a comprehensive library of modular components commonly used in modern AI architectures, including:
- **Attention Mechanisms**: Multi-query attention, sigmoid attention, flash attention, and more
- **Mixture of Experts (MoE)**: Efficient expert routing and gating mechanisms
- **Neural Network Modules**: Feedforward networks, activation functions, normalization layers
- **Quantization**: BitLinear, dynamic quantization, and other optimization techniques
- **Architectures**: Transformers, encoders, decoders, vision transformers, and complete model implementations
- **Training Utilities**: Optimization algorithms, logging, and performance monitoring
Each component is designed to be:
- **Modular**: Drop-in replacements that work seamlessly with PyTorch
- **High-Performance**: Optimized implementations with fused kernels where applicable
- **Well-Tested**: Comprehensive test coverage ensuring reliability
- **Production-Ready**: Used in hundreds of models across various domains
## Installation
```bash
pip3 install -U zetascale
```
## Quick Start
### Multi-Query Attention
Multi-query attention reduces memory usage while maintaining model quality by sharing key and value projections across attention heads.
```python
import torch
from zeta import MultiQueryAttention
# Initialize the model
model = MultiQueryAttention(
dim=512,
heads=8,
)
# Forward pass
text = torch.randn(2, 4, 512)
output, _, _ = model(text)
print(output.shape) # torch.Size([2, 4, 512])
```
### SwiGLU Activation
The SwiGLU activation function applies a gating mechanism to selectively pass information through the network.
```python
import torch
from zeta.nn import SwiGLUStacked
x = torch.randn(5, 10)
swiglu = SwiGLUStacked(10, 20)
output = swiglu(x)
print(output.shape) # torch.Size([5, 20])
```
### Relative Position Bias
Relative position bias quantizes the distance between positions into buckets and uses embeddings to provide position-aware attention biases.
```python
import torch
from torch import nn
from zeta.nn import RelativePositionBias
# Initialize the module
rel_pos_bias = RelativePositionBias()
# Compute bias for attention mechanism
bias_matrix = rel_pos_bias(1, 10, 10)
# Use in custom attention
class CustomAttention(nn.Module):
def __init__(self):
super().__init__()
self.rel_pos_bias = RelativePositionBias()
def forExcerpt of 15,544 characters
Read on GitHub
Kye Gomez · Swarms · United States
225
127
Colombia
94
8
5
2
白雨 | James Brown
2
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c5acccbe122d0175, topic:pytorch, topic:tensorflow
matched fp:c5acccbe122d0175, topic:chatgpt