Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Text mining using tidy tools :sparkles::page_facing_up::sparkles:
| Date | Stars |
|---|---|
| 2026-07-24 | 1201 |
| 2026-07-25 | 1201 |
| 2026-07-28 | 1201 |
| 2026-07-30 | 1201 |
| 2026-08-06 | 1201 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<!-- README.md is generated from README.Rmd. Please edit that file -->
# tidytext: Text mining using tidy tools <img src="man/figures/tidytext.png" align="right" />
**Authors:** [Julia Silge](https://juliasilge.com/), [David
Robinson](http://varianceexplained.org/)<br/> **License:**
[MIT](https://opensource.org/licenses/MIT)
<!-- badges: start -->
[](https://github.com/juliasilge/tidytext/actions/workflows/R-CMD-check.yaml)
[](https://cran.r-project.org/package=tidytext)
[](https://app.codecov.io/gh/juliasilge/tidytext?branch=main)
[](https://zenodo.org/badge/latestdoi/22224/juliasilge/tidytext)
[](https://joss.theoj.org/papers/10.21105/joss.00037)
[](https://CRAN.R-project.org/package=tidytext)
[](https://CRAN.R-project.org/package=tidytext)
<!-- badges: end -->
Using [tidy data principles](https://doi.org/10.18637/jss.v059.i10) can
make many text mining tasks easier, more effective, and consistent with
tools already in wide use. Much of the infrastructure needed for text
mining with tidy data frames already exists in packages like
[dplyr](https://cran.r-project.org/package=dplyr),
[broom](https://cran.r-project.org/package=broom),
[tidyr](https://cran.r-project.org/package=tidyr), and
[ggplot2](https://cran.r-project.org/package=ggplot2). In this package,
we provide functions and supporting data sets to allow conversion of
text to and from tidy formats, and to switch seamlessly between tidy
tools and existing text mining packages. Check out [our
book](https://www.tidytextmining.com/) to learn more about text mining
using tidy data principles.
### Installation
You can install this package from CRAN:
``` r
install.packages("tidytext")
```
Or you can install the development version from GitHub with
[remotes](https://github.com/r-lib/remotes):
``` r
library(remotes)
install_github("juliasilge/tidytext")
```
### Tidy text mining example: the `unnest_tokens` function
The novels of Jane Austen can be so tidy! Let’s use the text of Jane
Austen’s 6 completed, published novels from the
[janeaustenr](https://cran.r-project.org/package=janeaustenr) package,
and transform them to a tidy format. janeaustenr provides them as a
one-row-per-line format:
``` r
library(janeaustenr)
library(dplyr)
original_books <- austen_books() |>
group_by(book) |>
mutate(line = row_number()) |>
ungroup()
original_books
#> # A tibble: 73,422 × 3
#> text book line
#> <chr> <fct> <int>
#> 1 "SENSE AND SENSIBILITY" Sense & Sensibility 1
#> 2 "" Sense & Sensibility 2
#> 3 "by Jane Austen" Sense & Sensibility 3
#> 4 "" Sense & Sensibility 4
#> 5 "(1811)" Sense & Sensibility 5
#> 6 "" Sense & Sensibility 6
#> 7 "" Sense & Sensibility 7
#> 8 "" Sense & Sensibility 8
#> 9 "" Sense & Sensibility 9
#> 10 "CHAPTER 1" Sense & Sensibility 10
#> # ℹ 73,412 more rows
```
To work with this as a tidy dataset, we need to restructure it into a
**one-token-per-row** format. The `unnest_tokens()` function is a way to
convert a dataframe with a text column to be one-token-per-row:
``` r
library(tidytext)
tidy_books <- original_books |>
unnest_tokens(word, text)
tidy_books
#> # A tibble: 725,064 × 3
#> book line word
#> <fct> Excerpt of 11,387 characters
Read on GitHub
583
25
7
7
6
3
3
Jim Hester · @netflix
3
2
2
2
2
1
1
1
Canada
1
1
1
1
Michael Chirico · United States
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:cd82cadabdd744e0, topic:natural-language-processing