Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions
| Date | Stars |
|---|---|
| 2026-07-24 | 1254 |
| 2026-07-25 | 1254 |
| 2026-07-28 | 1254 |
| 2026-07-30 | 1254 |
| 2026-07-31 | 1253 |
| 2026-08-06 | 1253 |
Today
— stars today
This week
-1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> <img src="./docs/figures/intellagent_logo.png" alt="IntellAgent Logo" width="600"> <p><i>Uncover Your Agent's Blind Spots</i></p> [](https://discord.gg/YWbT87vAau) [](https://github.com/plurai-ai/intellagent/blob/main/LICENSE) [Documentation](https://intellagent-doc.plurai.ai/) | [Quick Start](#fire-quickstart) | [Newsletter](https://plurai.substack.com/) | [Paper](https://arxiv.org/pdf/2501.11067) </div> Simulate interactions, analyze performance, and gain actionable insights for conversational agents. Test, evaluate, and optimize your agent to ensure reliable real-world deployment. IntellAgent is an advanced multi-agent framework that transforms the evaluation and optimization of conversational agents. By simulating thousands of realistic, challenging interactions, IntellAgent stress-tests agents to uncover hidden failure points. These insights enhance agent performance, reliability, and user experience. ### Key Features - 🔬 **Generate Thousands of Edge-Case Scenarios:** Automatically generate highly realistic edge-case scenarios tailored specifically to your agent. - 🤖 **Simulate Diverse User Interactions:** Evaluate your agent across a wide spectrum of scenarios with varying complexity levels. - 📊 **Comprehensive Performance Evaluations:** Access detailed analysis to identify performance gaps, prioritize improvements, and compare outcomes across experiments. - 💪 **Simple integration:** Simple integration to your conversational agent. ## How it works  IntellAgent framework consists of three steps: - Given the user prompt (and optional additional information such as tools and database schema) - The system decomposes the prompt into a policy graph. - It samples a subset of policies based on their concurrence in real conversation distributions. - It generates a scenario of user-chatbot interaction (including system databases) to address the selected subset of policies. - Simulating the user-chatbot interaction using a user agent. - Critiquing the conversation and providing feedback on the tested policies. > To better understand the key concepts and how the IntellAgent system operates, refer to the [system overview guide](https://intellagent-doc.plurai.ai/How_it_Works/how-it-works/) ## 🔍 Demo  ## :fire: Quickstart > For a more detailed and comprehensive guide, see the [Start Guide](https://intellagent-doc.plurai.ai/quick_start/installation/). IntellAgent requires `python >= 3.9` <br /> #### Step 1 - Download and install ```bash git clone [email protected]:plurai-ai/intellagent.git cd intellagent ``` You can use Conda or pip to install the dependencies. Using pip: ```bash pip install -r requirements.txt ``` #### Step 2 - Set your LLM API Key Edit the `config/llm_env.yml` file to set up your LLM configuration (OpenAI/Azure/Vertex/Anthropic): ```yaml openai: OPENAI_API_KEY: "your-api-key-here" ``` To change the default LLM provider or model for either the IntellAgent system or the chatbot, you can easily update the configuration file. For instance, modify the `config/config_education.yml` file: ```yaml llm_intellagent: type: 'azure' llm_chat: type: 'azure' ``` To change the number of samples in the database you should modify the `num_samples` in the config file: ```yaml dataset: num_samples: 30 ``` > **Tokens Usage** > > We invest lots of effort in minimizing the total cost of running the simulator > - Using the default parameters, the expected cost per sample is approximately $0.10 > - You can control expenses by modifying the `cost_limit` limit parameter in the config file > - We are working on leveraging user data which will significantly reduce the cost per sample #### Step 3 - Run the Simulato
Excerpt of 7,822 characters
Read on GitHub
242
54
4
3
Vishwanath Martur · Freelancing · India
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f4ac0fe39361f6c2, topic:llmops
matched fp:f4ac0fe39361f6c2, topic:synthetic-data
matched fp:f4ac0fe39361f6c2, topic:evaluation