Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Large Language Models Are Reasoning Teachers (ACL 2023)
| Date | Stars |
|---|---|
| 2026-07-31 | 345 |
| 2026-08-03 | 345 |
| 2026-08-06 | 345 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Large Language Models Are Reasoning Teachers
Official repository for [Large Language Models Are Reasoning Teachers](https://arxiv.org/abs/2212.10071), by
Namgyu Ho, Laura Schmid, and Se-young Yun.
**🚀 Accepted to ACL 2023.**
This repository contains code for (1) running CoT reasoning on OpenAI models,
and (2) apply Fine-tune-CoT to train students based on OpenAI models *or* custom open-source models such as T5, Flan-T5, GPT-2 on your GPUs, based on 🤗 and Pytorch Lightning.
## Getting Started
### OpenAI API Experiments
OpenAI API experiments are implemented in the `oai` module. Refer to `notebooks/example_oai_finetune_cot.ipynb`
on how to run Fine-tune-CoT from start to finish.
### Updates to OpenAI API
Use the [Batch API](https://platform.openai.com/docs/guides/batch) to save costs for reasoning data collection, or use [parallel API requests](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py) to accelerate immediate, large-scale data collection.
### Custom Experiments (on GPU)
Custom experiments are implemented in the `custom` module, based on PyTorch Lightning. Refer to `custom_train.py`
and `scripts/custom/*.sh` on how to fine-tune models such as T5, Flan-T5, and GPT-2 using Fine-tune-CoT.
## Setup
```
pip install -r requirements.txt
python setup.py develop
```
### Environment
The code has been tested on Python<=3.10, PyTorch Lightning<=1.9, PyTorch>=2.0
## Data 🚀
We're proud to share *all* of our raw experimental data! All data is organized in json or jsonl format, for your pleasure :)
Cloud storage folder links:
- [Dropbox](https://www.dropbox.com/sh/hwcncpyomx87h20/AACqgVdd-ZzBQ3ncJcKqw0cVa?dl=0)
- [Google Drive](https://drive.google.com/drive/folders/1C6kah3WV36N8omlUl-TeU9tsJADZNaJV?usp=share_link)
### File List
- `dataset.tar.gz`: 12 task datasets compiled in a unified json format
- Belongs in `PROJECT/data/dataset/`
- `completion_data.tar.gz`: Completion data, i.e., inference data, from all teachers and students, for *all* experiments. About 8GB when uncompressed
- Belongs in `PROJECT/saved/completion_data/`
- `teacher_completion_data.tar.gz`: Completion data from Zero-shot-CoT (with diverse reasoning) on the default teacher model `text-davinci-002` using the OpenAI API. About 💰 $1000+ worth of goods, with ❤️ from [OSI LAB](http://osi.kaist.ac.kr) at [KAIST](https://kaist.ac.kr) . Subset of `completion_data.tar.gz`.
- Belongs in `PROJECT/saved/completion_data/`.
- `finetune_data.tar.gz`: *All* data used to fine-tune OpenAI students via the fine-tuning API, in jsonl format. These are derived from teacher completion data and can be generated from our code.
- Belongs in `PROJECT/saved/finetune_data/`
### Generate Paper Results
After downloading the full `completion_data.tar.gz`, you can run `notebooks/results.ipynb` to generate *all* result tables and figures from our paper. The code will (re-)evaluate all raw text model outputs contained in the completion data.
## Additional Resources
### Template-based Split (Paper Appendix E.3)
Template-based splits for MultiArith and Date Understanding are saved in `/data/splits/*__template.json`
### Few-shot Prompts
Few-shot prompts adapted from Wei 2022 are saved in `/data/few_shot_cot_prompts.json`
## Data Structures
### `data.dataset.Dataset`
```json
{
"metadata": {
"dataset_key": "multiarith"
},
"data": [
{
"sample_index": 0,
"question": "string",
"answer": "string",
"rationale": "string?"
}
]
}
```
### `data.completion.CompletionDataset`
```json
{
"metadata": {
"dataset_key": "multiarith",
"base_model": "curie",
"finetune_key": "zs_cot_multiarith",
"train_key": "ft_cot",
"prediction_template": "ft_cot_token",
},
"data": {
"<sample_index>": [
{
"sample_index": 0,
"completion_index": 0,
"question": "string",
"answer": "string",
"prompt": "string",
"completioExcerpt of 5,347 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:3be4a6a29b050372, llm:Repository title and description: 'Large Language Models Are Reasoning Teachers (ACL 2023)' — likely code/data accompanying an NLP paper about using LLMs to teach reasoning. Language: Jupyter Notebook. No topics provided.
matched fp:3be4a6a29b050372, llm:Repository title and description: 'Large Language Models Are Reasoning Teachers (ACL 2023)' — likely code/data accompanying an NLP paper about using LLMs to teach reasoning. Language: Jupyter Notebook. No topics provided.
matched fp:3be4a6a29b050372, llm:Repository title and description: 'Large Language Models Are Reasoning Teachers (ACL 2023)' — likely code/data accompanying an NLP paper about using LLMs to teach reasoning. Language: Jupyter Notebook. No topics provided.