Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
ETL, document parsing, web crawling, synthetic data generation, annotation and dataset curation tooling.
Signals: data-engineering, etl, data-pipeline, web-scraping, crawler, data-labeling, annotation-tool, synthetic-data
466 results
An AI web scraper using ollama, brightdata, selenium and other libraries.
AI-native data platform: 60+ connectors, dlt-based ETL, DAG scheduling, per-source RAG knowledge bases, and sandboxed AI agents that query, chart, and analyze your data.
🇰🇷 한국 정부 지원사업 전수조사 에이전트 스킬 [Claude Code·Codex·agy(Antigravity)·Cursor·Gemini CLI·Grok Build 지원] K-Startup·기업마당·NIPA·KOCCA·SMTECH 크롤링 + 즉시/로드맵/변형 3분류 (Korea-only)
A Claude Skill that automatically analyzes uploaded CSV files — generating summary statistics, detecting missing data, and creating quick visualizations using Python and pandas.
Parse incomplete JSON text in best-effort manner. Useful for partial JSON responses, broken network packages, LLM responses with markdown fences or exceeding token limits, and configuration files with comments.
Build data processing and data analysis pipelines that leverage the power of LLMs 🧠
Use LLMs to robustly extract web data
LLM-powered lossless compression tool
A package for parsing PDFs and analyzing their content using LLMs.
Filter sensitive information from free text before sending it to external services or APIs, such as chatbots and LLMs.
Convert Java to JSON/TOON and back. Supports complex object graphs, cyclic references, and TOON format for 40-50% LLM token savings
clean & curate your data with LLMs.
Parse PDFs into markdown using Vision LLMs
smart-llm-loader is a lightweight yet powerful Python package that transforms any document into LLM-ready chunks. Spend less time on preprocessing headaches and more time building what matters. From RAG systems to chatbots to document Q&A, SmartLLMLoader handles the heavy lifting so you can focus on creating exceptional AI applications.
A live reading list for LLM data synthesis (Updated to July, 2025).
DSIR large-scale data selection framework for language model training
Powerful RDF knowledge graph generation with RML mappings
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
Turn an entire GitHub Repo into a single organized .txt file to use with LLM's (ChatGPT, Claude, Gemini, etc)
🔍 Better text detection by combining multiple OCR engines (EasyOCR, Tesseract, and Pororo) with 🧠 LLM.
DOM to Semantic-Markdown for use with LLMs
DataInfra Series. Redact EVERYTHING with local llms and vlms.
PDF to markdown using vision LLMs — tables, layouts, and structure preserved
Twitter data scraping, embedding based image search and more.
24,535 repositories in the index in total.
