Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
| Date | Stars |
|---|---|
| 2026-07-24 | 537 |
| 2026-07-25 | 537 |
| 2026-07-28 | 537 |
| 2026-07-30 | 537 |
| 2026-07-31 | 537 |
| 2026-08-06 | 537 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<img src="./parti.jpeg" width="450px"></img>
## Parti - Pytorch
Implementation of <a href="https://parti.research.google/">Parti</a>, Google's pure attention-based text-to-image neural network, in Pytorch. <a src="https://sites.research.google/parti/">Project Page</a>
This repository also contains working training code for <a href="https://ai.googleblog.com/2022/05/vector-quantized-image-modeling-with.html">ViT VQGan VAE</a>. It also contains some additional modifications for faster training from vision transformers literature.
<a href="https://www.youtube.com/watch?v=qS-iYnp00uc">Yannic Kilcher</a>
Please join <a href="https://discord.gg/xBPBXfcFHd"><img alt="Join us on Discord" src="https://img.shields.io/discord/823813159592001537?color=5865F2&logo=discord&logoColor=white"></a> if you are interested in helping out with the replication with the <a href="https://laion.ai/">LAION</a> community
## Install
```bash
$ pip install parti-pytorch
```
## Usage
First you will need to train your Transformer VQ-GAN VAE
```python
from parti_pytorch import VitVQGanVAE, VQGanVAETrainer
vit_vae = VitVQGanVAE(
dim = 256, # dimensions
image_size = 256, # target image size
patch_size = 16, # size of the patches in the image attending to each other
num_layers = 3 # number of layers
).cuda()
trainer = VQGanVAETrainer(
vit_vae,
folder = '/path/to/your/images',
num_train_steps = 100000,
lr = 3e-4,
batch_size = 4,
grad_accum_every = 8,
amp = True
)
trainer.train()
```
Then
```python
import torch
from parti_pytorch import Parti, VitVQGanVAE
# first instantiate your ViT VQGan VAE
# a VQGan VAE made of transformers
vit_vae = VitVQGanVAE(
dim = 256, # dimensions
image_size = 256, # target image size
patch_size = 16, # size of the patches in the image attending to each other
num_layers = 3 # number of layers
).cuda()
vit_vae.load_state_dict(torch.load(f'/path/to/vae.pt')) # you will want to load the exponentially moving averaged VAE
# then you plugin the ViT VqGan VAE into your Parti as so
parti = Parti(
vae = vit_vae, # vit vqgan vae
dim = 512, # model dimension
depth = 8, # depth
dim_head = 64, # attention head dimension
heads = 8, # attention heads
dropout = 0., # dropout
cond_drop_prob = 0.25, # conditional dropout, for classifier free guidance
ff_mult = 4, # feedforward expansion factor
t5_name = 't5-large', # name of your T5
)
# ready your training text and images
texts = [
'a child screaming at finding a worm within a half-eaten apple',
'lizard running across the desert on two feet',
'waking up to a psychedelic landscape',
'seashells sparkling in the shallow waters'
]
images = torch.randn(4, 3, 256, 256).cuda()
# feed it into your parti instance, with return_loss set to True
loss = parti(
texts = texts,
images = images,
return_loss = True
)
loss.backward()
# do this for a long time on much data
# then...
images = parti.generate(texts = [
'a whale breaching from afar',
'young girl blowing out candles on her birthday cake',
'fireworks with blue and green sparkles'
], cond_scale = 3., return_pil_images = True) # conditioning scale for classifier free guidance
# List[PILImages] (256 x 256 RGB)
```
Realistically, when scaling up, you'll want to pre-encode your text into tokens and their respective mask
```python
from parti_pytorch.t5 import t5_encode_text
images = torch.randn(4, 3, 256, 256).cuda()
text_token_embeds, text_mask = t5_encode_text([
'a child screaming at finding a worm within a half-eaten apple',
'lizard running across the desert on two feet',
'waking up to a psychedelic landscape',
'seashells sparkling in the shallow waters'
], name = 't5-large', output_device = images.device)
# stoExcerpt of 8,202 characters
Read on GitHub
Phil Wang · United States
57
Pranoy · IIT Madras · India
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:5c9f67a64f4e5161, topic:text-to-image, desc:text-to-image, readme:text-to-image
matched fp:5c9f67a64f4e5161, topic:deep-learning