Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICML'24 Spotlight] "TravelPlanner: A Benchmark for Real-World Planning with Language Agents"
| Date | Stars |
|---|---|
| 2026-07-31 | 531 |
| 2026-08-06 | 532 |
Today
+1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h1 align="center">TravelPlanner<br> A Benchmark for Real-World Planning<br> with Language Agents </h1>
<p align="center">
<img src="images/icon.png" width="10%"> <br>
</p>
Code for the Paper "[TravelPlanner: A Benchmark for Real-World Planning with Language Agents](http://arxiv.org/abs/2402.01622)".
<p align="center">
[<a href="https://osu-nlp-group.github.io/TravelPlanner/">Website</a>] •
[<a href="http://arxiv.org/abs/2402.01622">Paper</a>] •
[<a href="https://huggingface.co/datasets/osunlp/TravelPlanner">Dataset</a>] •
[<a href="https://github.com/OSU-NLP-Group/TravelPlanner/blob/main/README.md#model-release">Models</a>] •
[<a href="https://huggingface.co/spaces/osunlp/TravelPlannerLeaderboard">Leaderboard</a>] •
[<a href="https://huggingface.co/spaces/osunlp/TravelPlannerEnvironment">Environment</a>] •
[<a href="https://twitter.com/ysu_nlp/status/1754365367294562680">Twitter</a>]
</p>
## Updates
- 2024/10/23: Release the [models](https://github.com/OSU-NLP-Group/TravelPlanner/blob/main/README.md#model-release) finetuned on TravelPlanner. The data could be found [here](./finetuning_data). We use [LLama-Factory](https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/train_lora/llama3_lora_sft.sh) for fine-tuning.
- 2024/7/14: Support [reference information](./database) in JSON format.
- 2024/4/28: Update the [warnings](https://github.com/OSU-NLP-Group/TravelPlanner/tree/main?tab=readme-ov-file#%EF%B8%8Fwarnings), please note that we strictly prohibit any form of cheating.
- 2024/4/21: Provide [format check tool](./postprocess/format_check.py) for testset submission files. You can run it to check if there are any format errors in your file.
# TravelPlanner
TravelPlanner is a benchmark crafted for evaluating language agents in tool-use and complex planning within multiple constraints.
For a given query, language agents are expected to formulate a comprehensive plan that includes transportation, daily meals, attractions, and accommodation for each day.
For constraints, from the perspective of real world applications, TravelPlanner includes three types of them: Environment Constraint, Commonsense Constraint, and Hard Constraint.
## Setup Environment
1. Create a conda environment and install dependencies:
```bash
conda create -n travelplanner python=3.9
conda activate travelplanner
pip install -r requirements.txt
```
2. Download the [database](https://drive.google.com/file/d/1pF1Sw6pBmq2sFkJvm-LzJOqrmfWoQgxE/view?usp=drive_link) and unzip it to the `TravelPlanner` directory (i.e., `your/path/TravelPlanner`).
## Running
### Two-stage Mode
In the two-stage mode, language agents are tasked with employing various search tools to gather information.
Based on the collected information, language agents are expected to deliver a plan that not only meets the user’s needs specified in the query but also adheres to commonsense constraints.
```bash
export OUTPUT_DIR=path/to/your/output/file
# We support MODEL in ['gpt-3.5-turbo-X','gpt-4-1106-preview','gemini','mistral-7B-32K','mixtral']
export MODEL_NAME=MODEL_NAME
export OPENAI_API_KEY=YOUR_OPENAI_KEY
# if you do not want to test google models, like gemini, just input "1".
export GOOGLE_API_KEY=YOUR_GOOGLE_KEY
# SET_TYPE in ['validation', 'test']
export SET_TYPE=validation
cd agents
python tool_agents.py --set_type $SET_TYPE --output_dir $OUTPUT_DIR --model_name $MODEL_NAME
```
The generated plan will be stored in OUTPUT_DIR/SET_TYPE.
### Sole-Planning Mode
TravelPlanner also provides an easier mode solely focused on testing their planning ability.
The sole-planning mode ensures that no crucialExcerpt of 10,213 characters
Read on GitHub
90
7
Kai Zhang · https://neocognition.io/ · United States
5
2
Google · Morocco
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f5c34c9a5a0c8abe, topic:large-language-models
matched fp:f5c34c9a5a0c8abe, topic:autonomous-agents