Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
| Date | Stars |
|---|---|
| 2026-07-24 | 746 |
| 2026-07-25 | 779 |
| 2026-07-28 | 821 |
| 2026-07-30 | 835 |
| 2026-08-06 | 835 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# AnakinScraper OSS
[](https://github.com/Anakin-Inc/anakinscraper-oss/actions/workflows/ci.yml)
[](LICENSE)
[](https://go.dev)
[](https://python.org)
[](docker-compose.yml)
[](webapp/)
The open-source web scraping API for AI. Turn any website into LLM-ready markdown or structured data.
Self-host with a single command. No cloud dependencies. Powers RAG pipelines, AI agents, and data extraction at scale.
```bash
git clone https://github.com/Anakin-Inc/anakinscraper-oss.git && cd anakinscraper-oss && make up
# Scrape any website — one curl, full result:
curl -s -X POST http://localhost:8080/v1/scrape \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}' | jq .markdown
```
## Why AnakinScraper?
| | AnakinScraper | Firecrawl | Crawlee | Scrapy |
|---|---|---|---|---|
| Anti-detect browser | Camoufox (Firefox) | Headless Chrome | Playwright | No |
| Smart proxy selection | Thompson Sampling (ML) | Round-robin | Manual | Manual |
| Zero-config start | `go run` — no DB needed | Docker required | npm install | pip install |
| Single binary | Go — one 30MB binary | Node.js | Node.js | Python |
| Handler chain fallback | HTTP → Browser → API | Single mode | Single mode | Single mode |
| Structured JSON (AI) | Gemini extraction | LLM extraction | No | No |
## Features
- **Handler chain with fallback** — HTTP fetch → anti-detect browser → external API. Each handler tries in order; if one fails, the next picks up automatically. Most pages resolve on the free local HTTP handler — paid APIs are only called for the ~5% that actually need them. [Docs →](docs/handlers.md)
- **Custom API handlers** — plug in any third-party scraping service as a chain fallback. Only invoked when local handlers fail — saves 90%+ on API costs vs routing everything through a paid service. Built-in [anakin.io](https://anakin.io) handler included. [How to add your own →](docs/handlers.md#example-adding-a-third-party-api-handler)
- **Domain configs** — per-domain scraping strategies: choose which handlers to use, set timeouts, retries, custom headers, block domains, and validate content with pattern matching. [Docs →](docs/domain-configs.md)
- **Failure detection** — define failure patterns and required patterns per domain. If the scraped content matches a failure pattern (e.g. CAPTCHA page) or misses a required pattern, the job auto-retries with the next handler. [Docs →](docs/domain-configs.md#failure-detection)
- **Anti-detect browser** — [Camoufox](https://github.com/daijro/camoufox) (anti-detect Firefox) with realistic fingerprints, not headless Chrome. [Docs →](docs/handlers.md#browser-handler)
- **Proxy auto-select** — [Thompson Sampling](https://en.wikipedia.org/wiki/Thompson_sampling) picks the best proxy per domain, learning from success/failure in real time. [Docs →](docs/proxy-pool.md)
- **Structured JSON extraction** — use Gemini AI to extract structured data from any page (bring your own API key)
- **Sync + async + batch API** — `POST /v1/scrape` for instant results, `/v1/url-scraper` for async with polling, batch up to 10 URLs
- **LLM-ready markdown** — automatic boilerplate removal, clean content extraction. Feed directly into RAG pipelines, Claude, GPT, or any LLM without preprocessing
- **Web dashboard** — built-in React UI for scraping, job tracking, domain config management, and proxy monitoring
- **Zero-config mode** — run with just Go, no database needed. Or use Docker for the full stack
- **Self-contained** — no Redis, Excerpt of 17,940 characters
Read on GitHub
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:65bd034d84e77b5d, topic:web-scraping, desc:web scraping, readme:web scraping
matched fp:65bd034d84e77b5d, topic:playwright