Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Pretrained ELECTRA Model for Korean
| Date | Stars |
|---|---|
| 2026-07-24 | 637 |
| 2026-07-25 | 637 |
| 2026-07-28 | 637 |
| 2026-07-30 | 637 |
| 2026-08-06 | 637 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
[한국어](./README.md) | [English](./README_EN.md)
# KoELECTRA
<p float="left" align="center">
<img width="900" src="https://user-images.githubusercontent.com/28896432/80024445-0f444e00-851a-11ea-9137-9da2abfd553d.png" />
</p>
[ELECTRA](https://openreview.net/pdf?id=r1xMH1BtvB)는 `Replaced Token Detection`, 즉 generator에서 나온 token을 보고 discriminator에서 "real" token인지 "fake" token인지 판별하는 방법으로 학습을 합니다. 이 방법은 모든 input token에 대해 학습할 수 있다는 장점을 가지며, BERT 등과 비교했을 때 더 좋은 성능을 보였습니다.
KoELECTRA는 **34GB의 한국어 text**로 학습하였고, 이를 통해 나온 `KoELECTRA-Base`와 `KoELECTRA-Small` 두 가지 모델을 배포하게 되었습니다.
또한 KoELECTRA는 **Wordpiece 사용**, **모델 s3 업로드** 등을 통해 OS 상관없이 `Transformers` 라이브러리만 설치하면 곧바로 사용할 수 있습니다.
## Download Link
| Model | Discriminator | Generator | Tensorflow-v1 |
| -------------------- | --------------------------------------------------------------------------------: | ------------------------------------------------------------------------: | --------------------------------------------------------------------------------------------------------------------: |
| `KoELECTRA-Base-v1` | [Discriminator](https://huggingface.co/monologg/koelectra-base-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-base-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-base-discriminator/blob/tfv1/koelectra-base-v1.tar.gz) |
| `KoELECTRA-Small-v1` | [Discriminator](https://huggingface.co/monologg/koelectra-small-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-small-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-small-discriminator/blob/tfv1/koelectra-small-v1.tar.gz) |
| `KoELECTRA-Base-v2` | [Discriminator](https://huggingface.co/monologg/koelectra-base-v2-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-base-v2-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-base-v2-discriminator/blob/tfv1/koelectra-base-v2.tar.gz) |
| `KoELECTRA-Small-v2` | [Discriminator](https://huggingface.co/monologg/koelectra-small-v2-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-small-v2-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-small-v2-discriminator/blob/tfv1/koelectra-small-v2.tar.gz) |
| `KoELECTRA-Base-v3` | [Discriminator](https://huggingface.co/monologg/koelectra-base-v3-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-base-v3-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-base-v3-discriminator/blob/tfv1/koelectra-base-v3.tar.gz) |
| `KoELECTRA-Small-v3` | [Discriminator](https://huggingface.co/monologg/koelectra-small-v3-discriminator) | [Generator](https://huggingface.co/monologg/koelectra-small-v3-generator) | [Tensorflow-v1](https://huggingface.co/monologg/koelectra-small-v3-discriminator/blob/tfv1/koelectra-small-v3.tar.gz) |
## About KoELECTRA
| | | Layers | Embedding Size | Hidden Size | # heads |
| ----------------- | ------------: | -----: | -------------: | ----------: | ------: |
| `KoELECTRA-Base` | Discriminator | 12 | 768 | 768 | 12 |
| | Generator | 12 | 768 | 256 | 4 |
| `KoELECTRA-Small` | Discriminator | 12 | 128 | 256 | 4 |
| | Generator | 12 | 128 | 256 | 4 |
### Vocabulary
- 이번 프로젝트의 가장 큰 목적은 **Transformers 라이브러리만 있으면 모델을 곧바로 사용 가능하게 만드는 것**이었고, 이에 Sentencepiece, Mecab을 사용하지 않고 원 논문과 코드에서 사용한 `Wordpiece`를 사용하였습니다.
- 자세한 내용은 [[Wordpiece Vocabulary]](./docs/wordpiece_vocab.md) 참고
| | Vocab Len | do_lower_case |
| --- | --------: | ------------: |
| v1Excerpt of 12,673 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:89fbf42e19b01532, topic:pytorch, topic:tensorflow
matched fp:89fbf42e19b01532, topic:language-model
matched fp:89fbf42e19b01532, topic:nlp
