Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena
| Date | Stars |
|---|---|
| 2026-07-31 | 799 |
| 2026-08-06 | 799 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# HyenaDNA  ## Important links: - [arxiv](https://arxiv.org/abs/2306.15794) - [blog](https://hazyresearch.stanford.edu/blog/2023-06-29-hyena-dna) - [colab](https://colab.research.google.com/drive/1wyVEQd4R3HYLTUOXEEQmp_I8aNC_aLhL?usp=sharing) - [huggingface](https://huggingface.co/LongSafari) - [discord](https://discord.gg/RJxUq4mzmW) - [youtube (talk)](https://youtu.be/haSkAC1fPX0?si=IUMmo_iGZ6SK1DBX) ## Intro Welcome to the HyenaDNA repo! HyenaDNA is a long-range genomic foundation model pretrained on context lengths of up to \***1 million tokens**\* at \***single nucleotide resolution**\*. The repo is a work in progress, but we're very excited to get this in the hands of researchers, so bare with us :) This repo is best suited for those who want to pretrain a HyenaDNA model, or try one of the downstream tasks from the paper. For the easiest entry point though, check out the HyenaDNA **[colab](https://colab.research.google.com/drive/1wyVEQd4R3HYLTUOXEEQmp_I8aNC_aLhL?usp=sharing)**, a self contained notebook that is Huggingface integrated. You'll be able to load pretrained weights and fine-tune on the GenomicBenchmarks dataset. Also, you'll be able to do inference and get embeddings on DNA sequences up to 450k nucleotides on the free tier. For 1 million long DNA sequences, you can get an A100 on Colab (paid tier), or run the notebook on your own machine. Credit: much of the code is forked and extended from [S4](https://github.com/HazyResearch/state-spaces) and [Safari](https://github.com/HazyResearch/safari). ## Discord Trying [Discord](https://discord.gg/RJxUq4mzmW) out! Maybe it'll be conducive to sharing ideas / tips on how HyenaDNA could be applied in different ways. Feel free to post questions there. ## Hugging Face pretrained weights <a name="huggingface"></a> Check these out :) There are different model sizes, and different training sequence lengths that they can handle up to. All pretrained on a single human reference genome (hg38). - [tiny-1k](https://huggingface.co/LongSafari/hyenadna-tiny-1k-seqlen/tree/main) - [tiny-1k-d256](https://huggingface.co/LongSafari/hyenadna-tiny-1k-seqlen-d256/tree/main) - [tiny-16k-d128](https://huggingface.co/LongSafari/hyenadna-tiny-16k-seqlen-d128/tree/main) - [small-32k](https://huggingface.co/LongSafari/hyenadna-small-32k-seqlen/tree/main) - [medium-160k](https://huggingface.co/LongSafari/hyenadna-medium-160k-seqlen/tree/main) - [medium-450k](https://huggingface.co/LongSafari/hyenadna-medium-450k-seqlen/tree/main) - [large-1m](https://huggingface.co/LongSafari/hyenadna-large-1m-seqlen/tree/main) See the suggested [GPU requirements](https://huggingface.co/LongSafari/hyenadna-tiny-1k-seqlen#hardware) for each model. There's a few way to use these HuggingFace weights, all with different flavors: 1. [colab](https://colab.research.google.com/drive/1wyVEQd4R3HYLTUOXEEQmp_I8aNC_aLhL?usp=sharing) 2. [Pytorch Lighting in this repo](#loadweights) 3. [standalone](#standalone) ## Dependencies <a name="dependencies"></a> For this repo, let's start with the dependancies that are needed. (If you're familiar with Docker, you can skip this section and jump to the [docker](#docker) setup below). The repo is built using Pytorch Lightning (a training library) and Hydra a config oriented ML library. (It'll be super helpful to get familiar with those tools.) - clone repo, cd into it ``` git clone --recurse-submodules https://github.com/HazyResearch/hyena-dna.git && cd hyena-dna ``` - create a conda environment, with Python 3.8+ ``` conda create -n hyena-dna python=3.8 ``` - The repo is developed with Pytorch 1.13, using cuda 11.7 ``` conda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.7 -c pytorch -c nvidia ``` - install requirements: ``` pip install -r requirements.txt ``` - install Flash Attention, these [notes](https://github.com/HazyResearch/safari#getting-started) will be helpf
Excerpt of 35,654 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c2ece5eab2f93529, topic:foundation-models
