Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
ETL, document parsing, web crawling, synthetic data generation, annotation and dataset curation tooling.
Signals: data-engineering, etl, data-pipeline, web-scraping, crawler, data-labeling, annotation-tool, synthetic-data
464 results
The GPT-based Universal Web Scraper MVP is a solution that leverages GPT models and web scraping libraries to generate scraper code based on user input and website analysis, simplifying the web scraping process.
Netflix like full-stack application with SPA client and backend implemented in service oriented architecture
High-performance web crawler API optimized for LLMs. Turn any search or website into clean Markdown using remote browsers. Firecrawl alternative
MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
Machine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)
A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one package
An unofficial API for Quora.
In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.
🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)
Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing.
🌐 Repositório com o conteúdo (slides, exemplos, códigos) da série de vídeos no YouTube sobre Web Scraping com Python.
OSINT tool to scrape names and usernames from large friend lists on Facebook, without being rate limited.
The unix-way web crawler
Python's package to scrap Twitter's front-end easily
A huge collection of awesome beginner-friendly Python projects starting from very basics to advance. Prefect repository for learning python and enhancing your python programming skills.
Library for Rapid (Web) Crawler and Scraper Development
Python CLI for Google Trends analysis with advanced rate limiting, cookie auth, and beautiful terminal output. Built on pytrends with enterprise features.
Crawl telegra.ph searching for nudes!
Scrape, standardize and share public meetings from local government websites
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username
Discover and verify professional emails using names + domains
24,545 repositories in the index in total.
