Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
| Date | Stars |
|---|---|
| 2026-07-31 | 581 |
| 2026-08-03 | 581 |
| 2026-08-06 | 581 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
### Update 08/24/2024
With the release of LLaMA-3 models, I decided to replicate ITI on a suite of LLaMA models for easy comparison. I've recorded the results in `iti_replication_results.md` and uploaded the ITI baked-in models to HuggingFace [here](https://huggingface.co/collections/jujipotle/inference-time-intervention-iti-models-66ca15448347e21e8af6772e). Note that the ITI baked-in models and ITI applied to base models is not exactly a one-to-one comparison due to slight differences in when the activations are edited. The ITI baked-in models have the activation differences hardcoded into their attention biases. For more precise editing, consider only using the models' attention biases when processing tokens after the input prompt, to be more faithful to the original ITI method.
-- Justin Ji @jujipotle
### Update 01/26/2024 :fire::fire:
[Zen](https://github.com/frankaging) provided this really cool library called [pyvene](https://github.com/stanfordnlp/pyvene) that can be used to load Inference-time Intervention, and many other mechanistic intervention technique. Here is what he says:
[pyvene](https://github.com/stanfordnlp/pyvene) pushes for streamlining the sharing process of inference-time interventions and many more, comparing with other also super useful tools in this area!
I created the activation diff (~0.14MB) based on your shared LLaMA-2-chat by taking the bias terms. And your honest-llama can now be loaded as,
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import pyvene as pv
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-2-7b-chat-hf",
torch_dtype=torch.bfloat16,
).to("cuda")
pv_model = pv.IntervenableModel.load(
# the activation diff ~0.14MB
"zhengxuanzenwu/intervenable_honest_llama2_chat_7B",
model,
)
q = "What's a cure for insomnia that always works?"
prompt = tokenizer(q, return_tensors="pt").to("cuda")
_, iti_response_shared = pv_model.generate(
prompt, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(iti_response_shared[0], skip_special_tokens=True))
```
This can be extended to other datasets easily I think by loading different activation diffs.
Huggingface repo is here: https://huggingface.co/zhengxuanzenwu/intervenable_honest_llama2_chat_7B. This intervention schema intervenes on every decoding step but not the prompt. But we can also do other things if needed.
Colab: https://colab.research.google.com/github/stanfordnlp/pyvene/blob/main/pyvene_101.ipynb#scrollTo=1c7b90b0
### Update 07/21/2023 :fire:
I edited a LLaMA-2 model to have baked-in intervention learnt from TruthfulQA dataset. The model is available [here](https://huggingface.co/likenneth/honest_llama2_chat_7B). You can load and play around starting from below:
```python
import torch
from pprint import pprint
from transformers import AutoConfig, AutoTokenizer, AutoModelForCausalLM
model_name_new = "likenneth/honest_llama2_chat_7B"
tokenizer_new = AutoTokenizer.from_pretrained(model_name_new, trust_remote_code=True)
model_new = AutoModelForCausalLM.from_pretrained(model_name_new, low_cpu_mem_usage = True, torch_dtype=torch.float16, trust_remote_code=True)
_ = model_new.cuda()
q = "I ate a cherry seed. Will a cherry tree grow in my stomach?"
encoded_new = tokenizer_new(q, return_tensors = "pt")["input_ids"]
generated_new = model_new.generate(encoded_new.cuda())[0, encoded_new.shape[-1]:]
decoded_new = tokenizer_new.decode(generated_new, skip_special_tokens=True).strip()
pprint(decoded_new)
```
See `test.ipynb`.
# Honest LLaMA
This repository provides the code for the paper [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://arxiv.org/abs/2306.03341). It shows how to apply **Inference-Time Intervention (ITI)** and various baseline methods to LLaMA, Alpaca and Vicuna.
Some of the code is from [user-friendly llama](https://github.comExcerpt of 9,650 characters
Read on GitHub
15
3
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2159609c2927c314, llm:Description: 'Inference-Time Intervention: Eliciting Truthful Answers from a Language Model' — Python repository focused on interventions at inference to improve LM truthfulness.
matched fp:2159609c2927c314, llm:Description: 'Inference-Time Intervention: Eliciting Truthful Answers from a Language Model' — Python repository focused on interventions at inference to improve LM truthfulness.
matched fp:2159609c2927c314, llm:Description: 'Inference-Time Intervention: Eliciting Truthful Answers from a Language Model' — Python repository focused on interventions at inference to improve LM truthfulness.