Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.
| Date | Stars |
|---|---|
| 2026-07-24 | 550 |
| 2026-07-25 | 550 |
| 2026-07-28 | 552 |
| 2026-07-30 | 552 |
| 2026-08-06 | 553 |
Today
+1 stars today
This week
+1 stars this week
This month
— stars this month
Momentum
5.0
growth rate 0.18%/day
<p align="center">
<img src="docs/assets/logo.png" alt="Reader Logo" width="200" />
</p>
<h1 align="center">Reader</h1>
<p align="center">
<strong>Open source web infrastructure for AI.</strong>
</p>
<p align="center">
Access the web without the complexity.
</p>
<p align="center">
<a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License: Apache 2.0"></a>
<a href="https://www.npmjs.com/package/@vakra-dev/reader"><img src="https://img.shields.io/npm/v/@vakra-dev/reader.svg" alt="npm version"></a>
<a href="https://github.com/vakra-dev/reader/stargazers"><img src="https://img.shields.io/github/stars/vakra-dev/reader.svg?style=social" alt="GitHub stars"></a>
</p>
<p align="center">
<a href="https://docs.reader.dev">Docs</a> · <a href="https://docs.reader.dev/home/examples">Examples</a> · <a href="https://discord.gg/6tjkq7J5WV">Discord</a>
</p>
<p align="center">
<img src="./docs/assets/demo.gif" alt="Reader demo - scrape any URL to clean markdown" width="700" />
</p>
## The Problem
Building agents that need web access is frustrating. You piece together Puppeteer, add stealth plugins, fight Cloudflare, manage proxies and it still breaks in production.
Because production grade web scraping isn't about rendering a page and converting HTML to markdown. It's about everything underneath:
| Layer | What it actually takes |
| ------------------------ | ------------------------------------------------------------------- |
| **Browser architecture** | Managing browser instances at scale, not one-off scripts |
| **Anti-bot bypass** | Cloudflare, Turnstile, JS challenges, they all block naive scrapers |
| **TLS fingerprinting** | Real browsers have fingerprints. Puppeteer doesn't. Sites know. |
| **Proxy infrastructure** | Standard vs premium, rotation strategies, sticky sessions |
| **Resource management** | Browser pooling, memory limits, graceful recycling |
| **Reliability** | Rate limiting, retries, timeouts, caching, graceful degradation |
I built **Reader**, a production-grade web scraping engine on top of Playwright, with stealth and anti-detection built in from the ground up.
## The Solution
Three primitives. That's it.
```typescript
import { ReaderClient } from "@vakra-dev/reader";
import { chromium } from "playwright-core";
const reader = new ReaderClient();
// 1. Scrape URLs → clean markdown
const result = await reader.scrape({ urls: ["https://example.com"] });
console.log(result.data[0].markdown);
// 2. Crawl a site → discover + scrape pages
const pages = await reader.crawl({
url: "https://example.com",
depth: 2,
scrape: true,
});
console.log(`Found ${pages.urls.length} pages`);
// 3. Browser session → full Playwright/Puppeteer control with stealth
const session = await reader.browser();
const browser = await chromium.connectOverCDP(session.wsEndpoint);
const page = browser.contexts()[0].pages()[0];
await page.goto("https://example.com");
console.log(await page.title());
await session.close();
```
All the hard stuff (browser pooling, anti-bot bypass, proxy rotation, retries) happens under the hood. You get clean markdown. Your agents get the web. And when you need full browser control, `browser()` gives you a stealthed Chrome that Playwright or Puppeteer can drive.
> [!TIP]
> If Reader is useful to you, a [star on GitHub](https://github.com/vakra-dev/reader) helps others discover the project.
## Features
- **Browser Sessions** - Launch stealthed Chrome, connect Playwright/Puppeteer via CDP
- **Anti-Bot Bypass** - TLS fingerprinting, navigator spoofing, WebRTC masking, `webdriver=false`
- **Clean Output** - Markdown and HTML with automatic main content extraction
- **Smart Content Cleaning** - Removes nav, headers, footers, popups, cookie banners
- **CLI & API** - Use from command line or progrExcerpt of 31,712 characters
Read on GitHub
45
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a8b04d82a8fe94ec, topic:web-scraping, topic:crawler, readme:web scraping