Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
ETL, document parsing, web crawling, synthetic data generation, annotation and dataset curation tooling.
Signals: data-engineering, etl, data-pipeline, web-scraping, crawler, data-labeling, annotation-tool, synthetic-data
466 results
Extract and convert data from any document, images, pdfs, word doc, ppt or URL into multiple formats (Markdown, JSON, CSV, HTML) with intelligent structured data extraction and advanced OCR.
The official repo for “Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting”, ACL, 2025.
OpenRecall is a fully open-source, privacy-first alternative to proprietary solutions like Microsoft's Windows Recall. With OpenRecall, you can easily access your digital history, enhancing your memory and productivity without compromising your privacy.
Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS :performing_arts:
Distributed crawler powered by Headless Chrome
A pytorch re-implementation of Real-time Scene Text Detection with Differentiable Binarization
PaddleOCR inference in PyTorch. Converted from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
Official implementation of Character Region Awareness for Text Detection (CRAFT)
超轻量级中文ocr，支持竖排文字识别, 支持ncnn、mnn、tnn推理 ( dbnet(1.8M) + crnn(2.5M) + anglenet(378KB)) 总模型仅4.7M
Bob 是一款 macOS 平台的翻译和 OCR 软件。
Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.
An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)
A web-scraping-based python package that enables you to scrape data from various platforms like GitHub, Twitter, Instagram, or any useful website.
Python Lambda Chrome Automation (naming pending)
本仓库收集整理爬虫相关资源，开发语言以Java为主
Example DAGs using hooks and operators from Airflow Plugins
ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.
NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。
📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )
🍁 Sycamore is an LLM-powered search and analytics platform for unstructured data.
Flock: A Low-Cost Streaming Query Engine on FaaS Platforms
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
24,535 repositories in the index in total.
