Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
clean & curate your data with LLMs.
| Date | Stars |
|---|---|
| 2026-07-31 | 487 |
| 2026-08-06 | 487 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# databonsai <img width="64" height="64" src="https://img.icons8.com/external-justicon-flat-justicon/64/external-bonsai-tree-justicon-flat-justicon.png" alt="external-bonsai-tree-justicon-flat-justicon"/>
[](https://badge.fury.io/py/databonsai)
[](https://opensource.org/licenses/MIT)
[](https://pypi.org/project/databonsai/)
[](https://github.com/psf/black)
## Clean & curate your data with LLMs
databonsai is a Python library that uses LLMs to perform data cleaning tasks.
## Features
- Suite of tools for data processing using LLMs including categorization,
transformation, and extraction
- Validation of LLM outputs
- Batch processing for token savings
- Retry logic with exponential backoff for handling rate limits and transient
errors
## Installation
```bash
pip install databonsai
```
Store your API keys on an .env file in the root of your project, or specify it
as an argument when initializing the provider.
```bash
OPENAI_API_KEY=xxx # if you use OpenAiProvider
ANTHROPIC_API_KEY=xxx # If you use AnthropicProvider
```
## Quickstart
### Categorization
Setup the LLM provider and categories (as a dictionary.
```python
from databonsai.categorize import MultiCategorizer, BaseCategorizer
from databonsai.llm_providers import OpenAIProvider, AnthropicProvider
provider = OpenAIProvider() # Or AnthropicProvider(). Highly recommend using Haiku, which is the default AnthropicProvider() model, as it is cheap and effective for these tasks
categories = {
"Weather": "Insights and remarks about weather conditions.",
"Sports": "Observations and comments on sports events.",
"Politics": "Political events related to governments, nations, or geopolitical issues.",
"Celebrities": "Celebrity sightings and gossip",
"Others": "Comments do not fit into any of the above categories",
"Anomaly": "Data that does not look like comments or natural language",
}
few_shot_examples = [
{"example": "Big stormy skies over city", "response": "Weather"},
{"example": "The team won the championship", "response": "Sports"},
{"example": "I saw a famous rapper at the mall", "response": "Celebrities"},
]
```
Categorize your data:
```python
categorizer = BaseCategorizer(
categories=categories,
llm_provider=provider,
examples = few_shot_examples,
#strict = False # Default true, set to False to allow for categories not in the provided dict
)
category = categorizer.categorize("It's been raining outside all day")
print(category)
```
Output:
```python
Weather
```
Use categorize_batch to categorize a batch. This saves tokens as it only sends
the schema and few shot examples once! (Works best for better models. Ideally,
use at least 3 few shot examples.)
```python
categories = categorizer.categorize_batch([
"Massive Blizzard Hits the Northeast, Thousands Without Power",
"Local High School Basketball Team Wins State Championship After Dramatic Final",
"Celebrated Actor Launches New Environmental Awareness Campaign",
])
print(categories)
```
Output:
```python
['Weather', 'Sports', 'Celebrities']
```
### AutoBatch for Larger datasets
If you have a pandas dataframe or list, use `apply_to_column_autobatch`
- Batching data for LLM api calls saves tokens by not sending the prompt for
every row. However, too large a batch size / complex tasks can lead to
errors. Naturally, the better the LLM model, the larger the batch size you
can use.
- This batching is handled adaptively (i.e., it will increase the batch size
if the response is valid and reduce it if it's not, with a decay factor)
Other features:
- Progress bar
- Returns the last successful index so you can resume froExcerpt of 7,655 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c400b5f24b8b26ef, llm:Repository description: "clean & curate your data with LLMs." Language: Python. No topics provided.
matched fp:c400b5f24b8b26ef, llm:Repository description: "clean & curate your data with LLMs." Language: Python. No topics provided.
matched fp:c400b5f24b8b26ef, llm:Repository description: "clean & curate your data with LLMs." Language: Python. No topics provided.