Website · Docs · Discord · Changelog
Connect the agent you are building, share the link with your team, and get structured feedback from the people who know the right answers.
Most tools start with tests or traces. Rhesis starts one step earlier: the people who know what the agent should answer are not the people building it, and their feedback rarely arrives in a form you can act on.
- Feedback that stays attached — Every review sits on the test case and the agent version that produced it, not in a Slack thread
- One pass/fail bar — The whole team reviews against the same tests, and you measure against them
- UI for reviewers, SDK and MCP for builders — Same data, three ways in
- From feedback to CI — Recurring feedback becomes tests and metrics that run on every change
1. Connect the agent you are building. If it already serves a public REST endpoint, paste the URL and you are done. Otherwise, and for most projects, use the SDK connector: your process opens an outbound WebSocket, so the agent works from your laptop or your VPC with no public URL.
2. Share the Rhesis link with your stakeholders. Domain experts, product managers, and reviewers open it in a browser. Nothing to install, no code.
3. They put the agent to work. In the playground they chat with the live agent and turn interesting conversations into tests. They create and run test sets, simulate multi-turn conversations, and leave structured feedback on what came back: pass/fail verdicts and comments, down to the individual metric or conversation turn.
4. Pull that feedback back into development. Read it from the SDK or the REST API, or work with it from Cursor, Claude Code, and other MCP clients through the Rhesis skill. Fix the agent, run the same tests again.
5. Agree on what the agent has to get right. Each cycle, feedback that arrived as prose becomes tests and metrics that check the same thing automatically. Reviews on a handful of cases end up as evals that run on every change.
| Role | How they use Rhesis |
|---|---|
| AI engineers | Connect the agent, pull feedback and reviewed test sets into the SDK, CI, and MCP while you build |
| Domain experts | Try the agent, review its answers, say what is wrong and what a correct answer looks like. UI, no code |
| Product managers | Turn scattered feedback into tests, and see whether the agent improves against the PRD. UI or MCP |
Nobody can review every case by hand. Once the first feedback is in, you can grow coverage from it:
- Test generation from your requirements, a PRD, or an uploaded file
- Conversation simulation with Penelope; adversarial probing with Polyphemus and garak
- 60+ metrics — RAGAS, DeepEval, garak, and custom LLM-as-Judge evaluators
- Traces linked to test results via OpenTelemetry
Generated tests are only as good as the requirements behind them. Instead of retyping a spec into a prompt, connect the tools your requirements already live in (Notion, GitHub, Jira, Confluence) and Rhesis writes tests from the real thing. See Tools.
| Use case | What you validate |
|---|---|
| Conversational & support agents | Role adherence, policy citation, escalation |
| RAG / document Q&A | Faithfulness, grounding, retrieval quality |
| Tool-using & multi-agent systems | Tool choice, goal completion, handoffs |
| Regulated / high-stakes domains | Must and must-not behaviors your reviewers defined |
Details: docs.rhesis.ai
Rhesis needs a way to invoke the agent under test. Two ways to do it: we recommend the SDK connector, but if your agent already has a public REST endpoint, that is the fastest way to get started.
Your process opens a persistent outbound WebSocket. Rhesis sends test inputs down that connection, so the agent needs no public URL and can stay on your laptop or inside your VPC. You write a function instead of describing a payload, and the same SDK carries tracing.
from rhesis.sdk.decorators import endpoint
@endpoint(name="my-chatbot")
def chat(message: str) -> str:
# Call your agent here
return my_agent(message)Run it, and the endpoint registers itself in Rhesis. See the SDK README for install, environments, and tracing.
Already serving HTTP on a reachable URL? Register it in the UI, no code and nothing to deploy. You supply auth headers plus request and response mapping, or let Rhesis derive the configuration from an OpenAPI spec or by exploring the endpoint. See Creating endpoints.
Either way, the next step is the same: share the link, and let your team start using the agent through the playground and test runs.
Feedback lands in Rhesis, but you do not have to leave your editor to act on it.
Use Rhesis from MCP-capable clients (Cursor, Claude Code, and others): design suites, pull sets and results, trigger runs, and read the feedback in the tools you already work in. Install with the skills CLI:
npx skills add rhesis-ai/rhesisSee the skills README.
Pull test runs, results, and the reviews attached to them from Python, or hit the API directly from CI in any language: OpenAPI spec.
Tracing (OpenTelemetry) and LLM providers for synthesis and judges are documented separately. See Tracing and Models. Neither is how you connect the agent under test.
app.rhesis.ai — managed service, connect your agent and invite your team.
git clone https://github.com/rhesis-ai/rhesis.git && cd rhesis && ./rh start./rh start pulls prebuilt images from GHCR. To build from the repo instead, use ./rh start --build (and ./rh restart --build after local Dockerfile changes).
Access: Frontend at localhost:3000, API at localhost:8080/docs
Commands: ./rh logs · ./rh stop · ./rh restart · ./rh delete
This setup enables auto-login for local testing. For production self-hosting, see Deployment docs.
Once the platform is running, connect your agent with the SDK:
pip install rhesis-sdkSee sdk/README.md.
| Option | Best for |
|---|---|
| Rhesis Cloud | Managed deployment |
Local Docker (./rh start) |
Development and trying the platform |
| Self-hosted | Production deployment — docs |
| Path | What it covers |
|---|---|
sdk/ |
Python SDK — connector, synthesizers, metrics, tracing |
skills/rhesis/ |
Agent skill + MCP workflows for Cursor, Claude Code, and others |
apps/backend/ |
API and workers |
apps/frontend/ |
Web UI |
docs/ |
Documentation site source |
MIT licensed. No plans to relicense core features. Enterprise features live in ee/ and remain separate.
We built Rhesis because the feedback that mattered most kept getting stuck outside the development loop. If you face the same problem, contributions are welcome.
See CONTRIBUTING.md for guidelines.
Ways to contribute: Fix bugs or add features · Contribute test sets for common failure modes · Improve documentation · Help others in Discord or GitHub discussions
- Documentation — Guides and API reference
- Discord — Community support
- GitHub Issues — Bug reports and feature requests
We take data security seriously. See our Privacy Policy for details.
Telemetry: Rhesis collects basic, anonymized usage statistics to improve the product. No sensitive data is collected or shared with third parties.
- Self-hosted: Opt out by setting
OTEL_RHESIS_TELEMETRY_ENABLED=false - Cloud: Telemetry enabled as part of Terms & Conditions
Made with 
in Potsdam, Germany 🇩🇪