Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Pretrained Language Models for Source code
| Date | Stars |
|---|---|
| 2026-07-31 | 258 |
| 2026-08-03 | 258 |
| 2026-08-18 | 259 |
| 2026-08-27 | 259 |
| 2026-09-19 | 259 |
| 2026-09-20 | 259 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<br/>
<h1 align="center">CodeTrans</h1>
<br/>
<br/>
[CodeTrans](https://github.com/agemagician/CodeTrans/) is providing **state of the art pre-trained models for source code**. CodeTrans was trained on **several Nvidia RTX 8000 GPUs** and **couple of Google TPUs** using various **State of the Art Transformers Models**.
Take a look into our paper [CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing](https://arxiv.org/abs/2104.02443) for more information about our work.
<br/>
<p align="center">
<img width="70%" src="https://cdn.pixabay.com/photo/2019/10/02/14/44/head-4521114_960_720.jpg" alt="CodeTrans Attention Visualization">
</p>
<br/>
This repository will be updated regulary with **new pre-trained models for source code** as part of supporting **software engineering** community in general, and **Source Code for Covid-19 research** specifically.
Table of Contents
=================
* [ ⌛️ Models Availability](#models)
* [ 🚀 Usage ](#usage)
* [ 🤖 Feature Extraction (FE)](#feature-extraction)
* [ 💥 Fine Tuning (FT)](#fine-tuning)
* [ 🧠 Prediction](#prediction)
* [ ⚗️ Source Code Generation ](#protein-generation)
* [ 🧐 Visualization ](#visualization)
* [ 📈 Benchmark ](#benchmark)
* [ 📊 Expected Results ](#results)
* [ 💻 Function Documentation Generation ](#function-documentation-generation)
* [ 💻 Source Code Summarization ](#source-code-summarization)
* [ 💻 Code Comment Generation ](#code-comment-generation)
* [ 💻 Commit Message Generation ](#commit-message-generation)
* [ 💻 API Sequence Recommendation ](#api-sequence-recommendation)
* [ 💻 Programming Language and Synthesis ](#programming-language-and-synthesis)
* [ ❤️ Community and Contributions ](#community)
* [ 📫 Have a question? ](#question)
* [ 🤝 Found a bug? ](#bug)
* [ ✅ Requirements ](#requirements)
* [ 🤵 Team ](#team)
* [ 💰 Sponsors ](#sponsors)
* [ 📘 License ](#license)
* [ ✏️ Citation ](#citation)
<a name="models"></a>
## ⌛️ Models Availability
All CodeTrans original Tensorflow checkpoints are downloadable from this [dropbox folder](https://www.dropbox.com/sh/488bq2of10r4wvw/AABrmE2V8lc8tRqV-qtLu4pUa?dl=0) and the pytorch checkpoints in the [Hugging Face model hub](https://huggingface.co/models?search=code_trans).
You can download all the datasets used in this research from [dropbox folder](https://www.dropbox.com/sh/mzxa2dq30gnot29/AABIf7wPxH5Oe0PZHJ5jPV22a?dl=0).
<a name="usage"></a>
## 🚀 Usage
How to use CodeTrans:
<a name="feature-extraction"></a>
* <b>🤖 Feature Extraction (FE):</b><br/>
coming soon.
<a name="fine-tuning"></a>
* <b>💥 Fine Tuning (FT):</b><br/>
coming soon.
<a name="prediction"></a>
* <b>🧠 Prediction:</b><br/>
Please check:
[Prediction Section](https://github.com/agemagician/CodeTrans/tree/master/prediction). More information coming soon.
<a name="code-generation"></a>
* <b>⚗️ Code Sequences Generation:</b><br/>
coming soon.
<a name="visualization"></a>
* <b>🧐 Visualization:</b><br/>
coming soon.
<a name="benchmark"></a>
* <b>📈 Benchmark:</b><br/>
coming soon.
<a name="results"></a>
## 📊 Expected Results
<a name="function-documentation-generation"></a>
* <b>💻 Function Documentation Generation (Bleu):</b><br/>
| Language / Model | Python | Java | Go | Php | Ruby | JavaScript |
| -------------------- | :------------: | :------------: | :------------: | :------------: | :------------: | :------------: |
| CodeTrans-ST-Small | 17.31 | 16.65 | 16.89 | 23.05 | 9.19 | 13.7 |
| CodeTrans-ST-Base | 16.86 | 17.17 | 17.16 | 22.98 | Excerpt of 13,881 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1f0d18e782165d9d, llm:Repository title and description: 'Pretrained Language Models for Source code' (CodeTrans) — focuses on pretrained models for source code tasks.
matched fp:1f0d18e782165d9d, llm:Repository title and description: 'Pretrained Language Models for Source code' (CodeTrans) — focuses on pretrained models for source code tasks.
matched fp:1f0d18e782165d9d, llm:Repository title and description: 'Pretrained Language Models for Source code' (CodeTrans) — focuses on pretrained models for source code tasks.