Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling
| Date | Stars |
|---|---|
| 2026-07-31 | 725 |
| 2026-08-02 | 725 |
| 2026-08-06 | 725 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# KAN-GPT
[](https://pypi.org/project/kan-gpt/)
[](https://codecov.io/gh/AdityaNG/kan-gpt)
[](https://github.com/AdityaNG/kan-gpt/actions/workflows/main.yml)
[](https://github.com/AdityaNG/kan-gpt/blob/main/LICENSE)
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling
## Install it from PyPI
```bash
pip install kan_gpt
```
## Citation
If you find our work useful cite us!
```
@misc{GANESH2024KANGPT,
author = {Aditya Nalgunda Ganesh},
title = {KAN-GPT: The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling},
year = {2024},
month = {May},
note = {Release 1.0.0, 9th May 2024},
url = {https://github.com/AdityaNG/kan-gpt/}
}
```
## Usage
Refer to the [KAN_GPT.ipynb](https://github.com/AdityaNG/kan-gpt/blob/main/KAN_GPT.ipynb) and [kan_gpt/prompt.py](https://github.com/AdityaNG/kan-gpt/blob/main/kan_gpt/prompt.py) for usage examples. The following is an outline of how to use the model:
```py
from kan_gpt.model import GPT
from transformers import GPT2Tokenizer
model_config = GPT.get_default_config()
model_config.model_type = "gpt2"
model_config.vocab_size = 50257
model_config.block_size = 1024
model = GPT(model_config)
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
prompt = "Bangalore is often described as the "
prompt_encoded = tokenizer.encode(
text=prompt, add_special_tokens=False
)
x = torch.tensor(prompt_encoded).unsqueeze(0)
model.eval()
y = model.generate(x, 50) # sample 50 tokens
result = tokenizer.decode(y[0])
print(result)
# Bangalore is often described as the Silicon Valley of India.
# The city has witnessed rapid growth in the past two decades.....
```
## Setup for Development
```bash
# Download Repo
git clone https://github.com/AdityaNG/kan-gpt
cd kan-gpt
git pull
# Download Dataset
python3 -m kan_gpt.download_dataset --dataset tinyshakespeare
python3 -m kan_gpt.download_dataset --dataset mnist
python3 -m kan_gpt.download_dataset --dataset webtext
# Install dependencies for development
pip install -r requirements.txt
pip install -e .
```
## Train
Use the following dummy script to make sure everything is working as expected
```bash
WANDB_MODE=offline CUDA_VISIBLE_DEVICE="" python3 -m kan_gpt.train --architecture MLP --batch_size 1 --dummy_dataset --device cpu --max_iters 200
WANDB_MODE=offline CUDA_VISIBLE_DEVICE="" python3 -m kan_gpt.train --architecture KAN --batch_size 1 --dummy_dataset --device cpu --max_iters 200
```
Then make use of the training script
```bash
python -m kan_gpt.train
```
## Prompt
You can prompt the model to produce text as follows
```bash
python -m kan_gpt.prompt --prompt "Bangalore is often described as the " --model_path (checkpoint)
```
## Results
We train and compare KAN-GPT with an equivalent MLP-GPT model on the Tiny Shakespeare dataset. We observe that the KAN-GPT performs slightly better than the MLP-GPT. We are looking into further experiments to dive deeper. The results are shown below:
| Metrics | | |
|---------|---------|---------|
|  |  |  |
## TODOs
- [x] Integrate [minGPT](https://github.com/karpathy/minGPT) and [pykan](https://github.com/KindXiaoming/pykan)
- [x] Dataset downloading script for [WebText](https://github.com/openai/gpt-2-output-dataset)
- [x] PyTorch Dataset parser for [WebText](https://github.com/openaExcerpt of 5,623 characters
Read on GitHub
83
5
1
Ikko Eltociear Ashimine · Japan
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:dbabbb8dd3b9c2e6, topic:llm, topic:gpt