Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Ask Me Anything language model prompting
| Date | Stars |
|---|---|
| 2026-07-31 | 548 |
| 2026-08-04 | 548 |
| 2026-08-06 | 548 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Ask Me Anything: A simple strategy for prompting language models  [](https://together.xyz/) This repository contains code for the Ask Me Anything (AMA) prompt-aggregation strategy. The end-to-end AMA approach includes (1) recursively using the language model to transform the task format and prompt and (2) aggregating the predictions of multiple prompts using weak supervision. We include code for both components and pointers to the publicly downloadable datasets. See our [paper](https://arxiv.org/abs/2210.02441) for more details. <p align="center"><img width="95%" src="imgs/decomp.png" /></p> ## Table of Contents - [Setup](#setup) - [Data](#getting-the-data) - [Running models](#models) - [Running experiments](#experiments) - [Repository Structure](#overall-repository-structure) - [Citation](#citation) ## Setup ### Installation Here we will setup the AMA code (prompting models for tasks), weak supervision code (aggregating predictions), and [Manifest](https://github.com/HazyResearch/manifest/) code (tooling for easily loading and running the models). We encourage the use of conda environments: ``` conda create --name ama python=3.8 conda activate ama ``` Clone as follows: ```bash # Ask Me Anything code git clone [email protected]:HazyResearch/ama_prompting.git cd ama_prompting pip install -r requirements.txt # Weak supervision code cd metal-ama git submodule init git submodule update pip install -e . # Manifest git clone [email protected]:HazyResearch/manifest.git cd manifest pip install -e . ``` ### Getting the data We assume all data lives in the ```AMA_DATA``` environment variable. By default, this is set to ```/home/data```. To change this, run ```bash export AMA_DATA=<path> ``` Please follow the instructions below to download all necessary data for experiments. 1. Download the PromptSource (P3) dataset from Hugging Face at https://huggingface.co/datasets/bigscience/P3. ```bash cd $AMA_DATA git lfs install git clone https://huggingface.co/datasets/bigscience/P3 ``` Then run [ama_prompting/download_p3.py](./download_p3.py). We use the GPT3-Style prompts in the few-shot baseline for each benchmark. 2. We downloaded the remaining tasks from the following sources: * [AGNews, DBPedia, and SST2](https://github.com/tonyzhaozh/few-shot-learning) * [Amazon Products](https://github.com/allenai/flex/blob/75d6d1cea66df2c8a7e3d429c6af5008ccf1544b/fewshot/hf_datasets_scripts/amazon/amazon.py) * [Natural Questions and WebQs](https://github.com/facebookresearch/FiD) * [RealTimeQA](https://github.com/realtimeqa/realtimeqa_public/tree/main/past/2022) (GCS files from June 17th - July 22, 2022) * [ReCoRD](https://sheng-z.github.io/ReCoRD-explorer/) * [StoryCloze](http://goo.gl/forms/aQz39sdDrO) ### Running models We run inference on models using a tool called [Manifest](https://github.com/HazyResearch/manifest). This tool is useful because it caches your inference results and does not require reloading the model for each new run you launch. To load the EleutherAI GPT-j-6B model, in a Tmux session, run: ```bash python3 manifest/manifest/api/app.py \ --model_type huggingface \ --model_name_or_path EleutherAI/gpt-j-6B \ --device 0 ``` It will take a few minutes for large models to load! To use a different model, replace ```EleutherAI/gpt-j-6B``` with the model name. See the Manifest repo for more information on loading other models. ## Experiments ### Collecting the prompting predictions To run a single task such as the Recognizing Textual Entailment (RTE) SuperGLUE benchmark, you can use the following steps. 1. Load a Manifest model using the above command 2. Run the following command. This will run the zero-shot baseline (```run_zeroshot = 1```), few-shot baseline (```run_fewshot = 1```) with $k$ in-context demonstrations (```k_shot = 3```), and th
Excerpt of 8,433 characters
Read on GitHub
Simran Arora
9
Laurel Orr · United States
3
Ikko Eltociear Ashimine · Japan
1
Avanika Narayan
1
Erjan K · Netherlands
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2b0840d0fd903553, llm:Repository name and description: 'Ask Me Anything language model prompting' (HazyResearch/ama_prompting). Likely prompts and prompting methods for LMs.