Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Pytorch implementation of R-BERT: "Enriching Pre-trained Language Model with Entity Information for Relation Classification"
| Date | Stars |
|---|---|
| 2026-08-02 | 362 |
| 2026-09-23 | 362 |
| 2026-11-04 | 362 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# R-BERT
[](https://paperswithcode.com/sota/relation-extraction-on-semeval-2010-task-8?p=enriching-pre-trained-language-model-with)
(Unofficial) Pytorch implementation of `R-BERT`: [Enriching Pre-trained Language Model with Entity Information for Relation Classification](https://arxiv.org/abs/1905.08284)
## Model Architecture
<p float="left" align="center">
<img width="600" src="https://user-images.githubusercontent.com/28896432/68673458-1b090d00-0597-11ea-96b1-7c1453e6edbb.png" />
</p>
### **Method**
1. **Get three vectors from BERT.**
- [CLS] token vector
- averaged entity_1 vector
- averaged entity_2 vector
2. **Pass each vector to the fully-connected layers.**
- dropout -> tanh -> fc-layer
3. **Concatenate three vectors.**
4. **Pass the concatenated vector to fully-connect layer.**
- dropout -> fc-layer
- **_Exactly the SAME conditions_** as written in paper.
- **Averaging** on `entity_1` and `entity_2` hidden state vectors, respectively. (including \$, # tokens)
- **Dropout** and **Tanh** before Fully-connected layer.
- **No [SEP] token** at the end of sequence. (If you want add [SEP] token, give `--add_sep_token` option)
## Dependencies
- perl (For evaluating official f1 score)
- python>=3.6
- torch==1.6.0
- transformers==3.3.1
## How to run
```bash
$ python3 main.py --do_train --do_eval
```
- Prediction will be written on `proposed_answers.txt` in `eval` directory.
## Official Evaluation
```bash
$ python3 official_eval.py
# macro-averaged F1 = 88.29%
```
- Evaluate based on the official evaluation perl script.
- MACRO-averaged f1 score (except `Other` relation)
- You can see the detailed result on `result.txt` in `eval` directory.
## Prediction
```bash
$ python3 predict.py --input_file {INPUT_FILE_PATH} --output_file {OUTPUT_FILE_PATH} --model_dir {SAVED_CKPT_PATH}
```
## References
- [Semeval 2010 Task 8 Dataset](https://drive.google.com/file/d/0B_jQiLugGTAkMDQ5ZjZiMTUtMzQ1Yy00YWNmLWJlZDYtOWY1ZDMwY2U4YjFk/view?sort=name&layout=list&num=50)
- [Semeval 2010 Task 8 Paper](https://www.aclweb.org/anthology/S10-1006.pdf)
- [NLP-progress Relation Extraction](http://nlpprogress.com/english/relationship_extraction.html)
- [Huggingface Transformers](https://github.com/huggingface/transformers)
- [https://github.com/wang-h/bert-relation-classification](https://github.com/wang-h/bert-relation-classification)
Excerpt of 2,559 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:415a6d90bee23868, topic:pytorch