Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
ETL, document parsing, web crawling, synthetic data generation, annotation and dataset curation tooling.
Signals: data-engineering, etl, data-pipeline, web-scraping, crawler, data-labeling, annotation-tool, synthetic-data
466 results
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
a stealthy browser automation framework
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
Collection of scripts corresponding to LucidProgramming YouTube tutorials
Faster requests on Python 3
A JavaScript library for generating random user agents with data that's updated daily.
HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.
Use Web Scraper API to extract data from Google Finance, including stock titles, pricing, and price changes in percentages.
The complete web scraping toolkit for PHP.
In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.
Simple web scraping for R
Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabs’ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
Web Scraping Framework
Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
A Chrome DevTools Protocol driver for web automation and scraping.
A lightweight stream processing library for Go
Lightweight document management system packed with all the features you can expect from big expensive solutions
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
Ybat - YOLO BBox Annotation Tool
24,535 repositories in the index in total.
