Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
| Date | Stars |
|---|---|
| 2026-07-24 | 397 |
| 2026-07-25 | 397 |
| 2026-07-28 | 397 |
| 2026-07-30 | 397 |
| 2026-08-06 | 397 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Web Scraping Reference: Cheat Sheet for Web Scraping using R
Inspired by Hartley Brody, this cheat sheet is about web scraping using [rvest](https://github.com/hadley/rvest),[httr](https://github.com/r-lib/httr) and [Rselenium](https://github.com/ropensci/RSelenium). It covers many topics in this [blog](https://blog.hartleybrody.com/web-scraping-cheat-sheet/).
While Hartley uses python's requests and beautifulsoup libraries, this cheat sheet covers the usage of httr and rvest. While rvest is good enough for many scraping tasks, httr is required for more advanced techniques. Usage of Rselenium(web driver) is also covered.
I also recommend the book [The Ultimate Guide to Web Scraping](https://blog.hartleybrody.com/guide-to-web-scraping/) by Hartley Brody. Though it uses Python libraries, the underlying logic of web scraping is the same. The same strategies can be applied using any languages including R.
Please post issues [here](https://github.com/yusuzech/r-web-scraping-cheat-sheet/issues) if you find any errors or have any recommendations.
# Table of Contents
1. <a href="#rvest">Web Scraping using rvest and httr</a>
1. <a href="#rvest1">Useful Libraries and Resources</a>
2. <a href="#rvest2">Making Simple Requests</a>
3. <a href="#rvest3">Inspecting Response</a>
4. <a href="#rvest4">Extracting Elements from HTML</a>
5. <a href="#rvest5">Storing Data in R</a>
1. <a href="#rvest5.1">Storing Data as list</a>
2. <a href="#rvest5.2">Storing Data as data.frame</a>
6. <a href="#rvest6">Saving Data to disk</a>
1. <a href="#rvest6.1">Saving Data to csv</a>
2. <a href="#rvest6.2">Saving Data to SQLite Database</a>
7. <a href="#rvest7">More Advanced Topics</a>
1. <a href="#rvest7.1">Javascript Heavy Websites</a>
2. <a href="#rvest7.2">Content Inside iFrames</a>
3. <a href="#rvest7.3">Sessions and Cookies</a>
4. <a href="#rvest7.4">Delays and Backing Off</a>
5. <a href="#rvest7.5">Spoofing the User Agent</a>
6. <a href="#rvest7.6">Using Proxy Servers</a>
7. <a href="#rvest7.7">Setting Timeouts</a>
8. <a href="#rvest7.8">Handling Network Errors</a>
9. <a href="#rvest7.9">Downloading Files</a>
10. <a href="#rvest7.10">Logins and Sessions</a>
11. <a href="#rvest7.11">Web Scraping in Parallel</a>
2. <a href="#rselenium">Web Scraping using Rselenium</a>
1. <a href="#rselenium1">Why RSelenium</a>
1. <a href="#rselenium1.1">Pros and Cons from Using RSelenium</a>
2. <a href="#rselenium1.2">Useful Resources</a>
2. <a href="#rselenium2">Interacting with the Web Driver In Rselenium</a>
1. <a href="#rselenium2.0">How to Start</a>
2. <a href="#rselenium2.1">Navigating to different URLs</a>
3. <a href="#rselenium2.2">Simulating Scrolls, Clicks, Text Inputs, Logins, and Other Actions</a>
3. <a href="#rselenium3">Extract Content from the Web Page</a>
1. <a href="#rselenium3.1">Extracting Content using Rselenium</a>
2. <a href="#rselenium3.2">Extracting Content using Parsed Page Source and `rvest`</a>
4. <a href = "#rselneium4">miscellanea</a>
1. <a href="#rselenium4.1">Javascript</a>
2. <a href="#rselenium4.2">Iframe</a>
3. <a href="#change">Change Log</a>
# 1. <a name="rvest">Web Scraping using rvest and httr</a>
## 1.1. <a name="rvest1">Useful Libraries and Resources</a>
[rvest](https://github.com/hadley/rvest) is built upon the xml2 package and also accept config from the `httr` package. For the most part, we only need `rvest`. However, we need `httr` if we want to add extra configurations.
To install those two packages:
```r
install.packages("rvest")
install.packages("httr")
```
To load them:
```r
require(rvest)
require(httr)
```
There are many resources available online; these are what I found to be the most useful:
1. [w3schools CSS selectors reference](https://www.w3schools.com/CSSref/css_selectors.asp) : if yExcerpt of 40,199 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1e92725affd05720, topic:web-scraping, name:web scraping, desc:web scraping