Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Tools for curating biomedical training data for large-scale language modeling
| Date | Stars |
|---|---|
| 2026-07-31 | 506 |
| 2026-08-04 | 507 |
| 2026-08-06 | 507 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# BigBIO: Biomedical Dataset Library
`BigBIO` (BigScience Biomedical) is an open library of biomedical dataloaders built using Huggingface's (🤗) [`datasets` library](https://huggingface.co/docs/datasets/) for data-centric machine learning.
Our goals include:
- Lightweight, programmatic access to biomedical datasets at scale
- Promoting reproducibility in data processing
- Better documentation for dataset provenance, licensing, and other key attributes
- Easier generation of meta-datasets for natural language prompting, multi-task learning
Currently `BigBIO` provides support for:
- 126+ biomedical datasets
- 10+ languages
- 12 task categories
- Harmonized dataset schemas by task type
- Metadata on *licensing*, *coarse/fine-grained task types*, *domain*, and more!
## How to Use `BigBIO`
The preferred way to use these datasets is to access them from the [Official `BigBIO` Hub](https://huggingface.co/bigbio).
Minimally, ensure you have the `datasets` library installed. Preferably, install the requirements as follows:
`pip install -r requirements.txt`.
<br>
You can access `BigBIO` datasets as follows:
```python
from datasets import load_dataset
data = load_dataset("bigbio/biosses")
```
In most cases, scripts load the original schema of the dataset by default. You can also access the `BigBIO` split that streamlines access to key information in datasets given a particular task.
<br>
For example, the `biosses` dataset follows a `pairs` based schema, where text-based inputs (sentences, paragraphs) are assigned a "translated" pair.
```python
from datasets import load_dataset
data = load_dataset("bigbio/biosses", name="biosses_bigbio_pairs")
```
Generally, you can load your datasets as follows:
```python
# Load original schema
data = load_dataset("bigbio/<your_dataset>")
# Load BigBIO schema
data = load_dataset("bigbio/<your_dataset_here>", name="<your_dataset>_bigbio_<schema_name>")
```
Check the datacards on the Hub to see what splits are available to you. You can find more information about [schemas](task_schemas.md) in [Documentation](##Documentation) below.
## Benchmark Support
`BigBIO` includes support for almost all datasets included in other popular English biomedical benchmarks.
| Task Type | Dataset | [`BigBIO` (ours)](https://arxiv.org/abs/2206.15076) | [BLUE](https://arxiv.org/abs/1906.05474) | [BLURB](https://microsoft.github.io/BLURB/) | [BoX](https://arxiv.org/abs/2204.07600) | DUA needed |
|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|
| NER | BC2GM | ✓ | | ✓ | ✓ | |
| NER | BC5-chem | ✓ | ✓ | ✓ | ✓ | |
| NER | BC5-disease | ✓ | ✓ | ✓ | ✓ | |
| NER | EBM PICO | ✓ | | ✓ | | |
| NER | JNLPBA | ✓ | | ✓ | ✓ | |
| NER | NCBI-disease | ✓ | | ✓ | ✓ | |
| RE | ChemProt | ✓ | ✓ | ✓ | ✓ | |
| RE | DDI | ✓ | ✓ | ✓ | ✓ | |
| RE | GAD | ✓ | | ✓ | | |
| QA | PubMedQA | ✓ | | ✓ | ✓ | |
| QA | BioASQ | ✓ | | ✓ | ✓ | ✓ |
| DC | HoC | ✓ | ✓ | ✓ | ✓ | |
| STS | BIOSSES | ✓ | ✓ | ✓ | | |
| STS | MedSTS | * | ✓ | | | ✓ |
| NER | n2c2 2010 | ✓ | ✓ | | ✓ | ✓ |
| NER | ShARe/CLEF 2013 | * | ✓ | | | ✓ |
| NLI | MedNLI | ✓ | ✓ | | | ✓ |
| NER | n2c2 deid 2006 | ✓ | | | ✓ | ✓ |
| DC | n2c2 RFHD 2014 | ✓ | | | ✓ | ✓ Excerpt of 7,749 characters
Read on GitHub
195
100
79
47
43
Simon Ott · Austria
25
23
Debajyoti Datta · United States
17
13
13
11
9
8
7
6
John Giorgi · @abridgeai
6
6
4
Stephen Bach · Brown University
4
4
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:08453223a763c31a, desc:training data