Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Persian NLP Toolkit
| Date | Stars |
|---|---|
| 2026-07-24 | 1412 |
| 2026-07-25 | 1412 |
| 2026-07-28 | 1412 |
| 2026-07-30 | 1412 |
| 2026-08-06 | 1412 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Hazm - Persian NLP Toolkit
[**Hazm**](https://www.roshan-ai.ir/hazm/) is a python library to perform natural language processing tasks on Persian text. It offers various features for analyzing, processing, and understanding Persian text. You can use Hazm to normalize text, tokenize sentences and words, lemmatize words, assign part-of-speech tags, identify dependency relations, create word and sentence embeddings, or read popular Persian corpora.
[](documentation/assets/sample.png)
## Features
- **Normalization:** Converts text to a standard form (diacritics removal, ZWNJ correction, etc).
- **Tokenization:** Splits text into sentences and words.
- **Lemmatization:** Reduces words to their base forms.
- **POS tagging:** Assigns a part of speech to each word.
- **Dependency parsing:** Identifies the syntactic relations between words.
- **Embedding:** Creates vector representations of words and sentences.
- **Hugging Face Integration:** Automatically download and cache pretrained models from the Hub.
- **Persian corpora reading:** Easily read popular Persian corpora with ready-made scripts.
## Installation
To install the latest version of Hazm (requires Python 3.12+), run:
pip install hazm
To use the pretrained models from Hugging Face, ensure you have the `huggingface-hub` package:
pip install huggingface-hub
## Pretrained-Models
Hazm supports automatic downloading of pretrained models. You can find all available models (POS Tagger, Chunker, Embeddings, etc.) on our official Hugging Face page:
👉 [**Roshan Research on Hugging Face**](https://huggingface.co/roshan-research/models)
When using Hazm, simply provide the `repo_id` and `model_filename` as shown in the examples below, and the library will handle the rest.
## Usage
```python
from hazm import *
# ===============================
# Stemming
# ===============================
stemmer = Stemmer()
stem = stemmer.stem('کتابها')
print(stem) # کتاب
# ===============================
# Normalizing
# ===============================
normalizer = Normalizer()
normalized_text = normalizer.normalize('من کتاب های زیــــادی دارم .')
print(normalized_text) # من کتابهای زیادی دارم.
# ===============================
# Lemmatizing
# ===============================
lemmatizer = Lemmatizer()
lem = lemmatizer.lemmatize('مینویسیم')
print(lem) # نوشت#نویس
# ===============================
# Sentence tokenizing
# ===============================
sentence_tokenizer = SentenceTokenizer()
sent_tokens = sentence_tokenizer.tokenize('ما کتاب میخوانیم. یادگیری خوب است.')
print(sent_tokens) # ['ما کتاب می\u200cخوانیم.', 'یادگیری خوب است.']
# ===============================
# Word tokenizing
# ===============================
word_tokenizer = WordTokenizer()
word_tokens = word_tokenizer.tokenize('ما کتاب میخوانیم')
print(word_tokens) # ['ما', 'کتاب', 'می\u200cخوانیم']
# ===============================
# Part of speech tagging
# ===============================
tagger = POSTagger(repo_id="roshan-research/hazm-postagger", model_filename="pos_tagger.model")
tagged_words = tagger.tag(word_tokens)
print(tagged_words) # [('ما', 'PRON'), ('کتاب', 'NOUN'), ('می\u200cخوانیم', 'VERB')]
# ===============================
# Chunking
# ===============================
chunker = Chunker(repo_id="roshan-research/hazm-chunker", model_filename="chunker.model")
chunked_tree = tree2brackets(chunker.parse(tagged_words))
print(chunked_tree) # [ما NP] [کتاب NP] [میخوانیم VP]
# ===============================
# Word embedding
# ===============================
word_embedding = WordEmbedding.load(repo_id='roshan-research/hazm-woExcerpt of 7,320 characters
Read on GitHub
328
131
90
69
40
30
20
19
18
8
7
5
4
4
Hassan Bashari · Iran
3
2
2
2
2
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:fba2f76bbdcf8fc1, topic:nlp, topic:natural-language-processing, topic:tokenizer