AgentsMeetRL is an awesome list that summarizes open-source repositories for training LLM Agents using reinforcement learning:
- 🤖 The criteria for identifying an agent project are that it must have at least one of the following: multi-turn interactions or tool use (so TIR projects, Tool-Integrated Reasoning, are considered in this repo).
⚠️ This project is based on code analysis from open-source repositories using LLM coding agents, which may contain unfaithful cases. Although manually reviewed, there may still be omissions. If you find any errors, please don't hesitate to let us know immediately through issues or PRs - we warmly welcome them!- 🚀 We particularly focus on the reinforcement learning frameworks, RL algorithms, rewards, and environments that projects depend on, for everyone's reference on how these excellent open-source projects make their technical choices. See [Click to view technical details] under each table.
- 📅 Last updated: 2026-07-23
- 🤗 Feel free to submit your own projects anytime - we welcome contributions!
- 📚 If you find this repository helpful for your research, please cite it via the "Cite this repository" button on the right sidebar.
Taxonomy:
- Base Framework: General-purpose RL training frameworks for LLM agents (e.g., veRL, OpenRLHF, trl)
- General/MultiTask: Agent systems trained/evaluated across multiple tasks or environments
- Search & RAG: Search-augmented reasoning agents that use retrieval tools to enhance LLM reasoning
- Web & GUI: Agents that interact with web browsers, mobile/desktop GUIs, or operating systems
- Tool-Use: Agents trained to invoke external tools (APIs, code executors, MCP, etc.)
- Code & SWE: Software engineering and code generation agents
- Reasoning: Reasoning agents with tool-integrated or multi-turn reasoning (math, QA, visual)
- Multi-Agent RL: Multi-agent collaboration, negotiation, or credit assignment via RL
- Memory: Agents that learn to manage, retrieve, or evolve memory
- Embodied: Agents operating in embodied/physical simulation environments
- Domain-Specific: RL agents for specialized domains (medical, OS tuning, etc.)
- Reward & Training: Process/outcome reward models and training methodologies for agents
- Safety: RL for agent safety alignment, adversarial red-teaming, and jailbreak defense/attack
- VLM Agent: Vision-language model agents trained with RL for multimodal interaction
- Self-Evolution: Agents that self-evolve via RL feedback loops (
⚠️ definition still evolving in the community) - Environment: Benchmarks, gyms, and sandbox environments for agent training/evaluation
Some Enumeration:
- Enumeration for Reward Type:
- External Verifier: e.g., a compiler or math solver
- Rule-Based: e.g., a LaTeX parser with exact match scoring
- Model-Based: e.g., a trained verifier LLM or reward LLM
- Custom
- 📢 2026-07 Update: Added 13 new repositories from late-Jun–Jul 2026 across 8 categories (Self-Evolution +3 [SEED/OPID/UCOB, the on-policy-distillation-for-agentic-RL line], VLM Agent +2 [VTS/VSeek, long-video search agents], Tool-Use +2 [Tool-RL-Box; plus catch-up of AWorld-RL], Environment +2 [SETA terminal envs, OpenAgent tool-generalization sandbox], Web & GUI +1 [SCALE-CUA], Embodied +1 [REAL], Memory +1 [Supersede], Domain-Specific +1 [FaithMed]). Every entry was verified by opening the repo and confirming real RL-training (or environment) code — papers whose code is not yet released (EvoCUA-1.5, DeepSearch-World, CompactionRL, GUICrafter, VideoSearcher, Xiaomi-GUI-0) were deliberately left out.
- 📢 2026-06 Update: Added 43 new repositories across 11 categories (VLM Agent +8, Search & RAG +7, Environment +6, Reward & Training +4, Base Framework/Tool-Use/Self-Evolution/Embodied +3 each, Web & GUI/Code & SWE/Domain-Specific +2 each). New since the last update: Harness-1, FastContext, OpenWebRL, Polar, AgentJet, HarnessX, APPO, SPADER, DeepRubric, Embodied-R1.5, SIRI; plus catch-up of earlier-2026 misses (Vision-DeepResearch, ARM-Thinker, PyVision-RL, Gen-Searcher, DataMind, Tool-R0, Agent World Model, VisGym, Gym-Anything, ChemCraft, OpAgent, etc.).
- 📢 2026-05 Update: Added 17 new repositories from Apr–May 2026 across 11 categories (notably General/MultiTask +4 [SkillZero/T²PO/SDAR/StraTA, mostly ZJU-REAL & related agentic RL methods], VLM Agent +3 [MTA-Agent/ParaVT/OpenSearch-VL, multimodal deep search & video tool use], Web & GUI +2 [ClawGUI/ToolCUA]). Moved CoEvolve to "Under Review" (code not yet released).
- 📢 2026-04 Update: Added 67 new repositories covering Apr 2025 – Apr 2026 across nearly every category (notably VLM Agent +9, Search & RAG +10, Web & GUI +7, Tool-Use +7). Also reclassified SkyRL (→ General) and SPIRAL (→ Multi-Agent), and updated the VAGEN entry to its NeurIPS'25 upstream repo.
- 📢 2026-03 Update: Restructured taxonomy from 12 to 16 categories (added Multi-Agent RL, Reward & Training, Safety, VLM Agent, Self-Evolution, Domain-Specific; merged GUI into Web & GUI; retired TextGame/Biomedical). Added ~70 new repositories covering Sep 2025 – Mar 2026, growing the total from ~134 to 205.
This list is also packaged as a Claude Code Skill — agents-meet-rl — that turns the corpus into an on-demand assistant for agentic-RL training, evaluation, and experiment design: reward not moving, KL / entropy / length blow-ups, GRPO / PPO / DAPO knobs, retokenization drift, tool-call parse failures, long-horizon credit assignment, LLM-judge inconsistency, benchmark contamination, and framework / benchmark / algorithm selection — each answer anchored to specific papers and repos from this list. Backed by a machine-readable corpus of 368 projects (snapshot 2026-07-23). Once installed, Claude Code auto-invokes it whenever your question matches.
Install as a plugin (recommended):
/plugin marketplace add thinkwee/claude-plugins
/plugin install agents-meet-rl@thinkwee
Or install manually:
git clone https://github.com/thinkwee/AgentsMeetRL
cp -r AgentsMeetRL/skills/agents-meet-rl ~/.claude/skills/Then just ask, e.g. "my GRPO search agent's reward is flat but eval keeps dropping" or "which RL framework should I pick for a multi-turn tool-use agent?" — the skill routes your symptom to fixes grounded in this corpus.
| Github Repo | 🌟 Stars | Date | Org | Paper Link |
|---|---|---|---|---|
| AgentJet |  |
2026.6 | ModelScope (Alibaba) | Paper |
| HarnessX |  |
2026.6 | Darwin-Agent | Paper |
| Polar |  |
2026.5 | NVIDIA (NeMo) | Paper |
| uni-agent |  |
2026.4 | verl-project | -- |
| VeRL-Omni |  |
2026.4 | verl-project | -- |
| OpenClaw-RL |  |
2026.3 | Gen-Verse | Paper |
| Claw-R1 |  |
2026.3 | USTC | -- |
| Open-AgentRL |  |
2026.2 | Gen-Verse | Paper |
| NeMo-RL |  |
2026.1 | NVIDIA | -- |
| RLinf |  |
2025.8 | Tsinghua/Infinigence AI/PKU | Paper |
| siiRL |  |
2025.7 | Shanghai Innovation Institute | Paper |
| slime |  |
2025.6 | Tsinghua University (THUDM) | blog |
| agent-lightning |  |
2025.6 | Microsoft Research | Paper |
| AReaL |  |
2025.6 | AntGroup/Tsinghua | Paper |
| ROLL |  |
2025.6 | Alibaba | Paper |
| MARTI |  |
2025.5 | Tsinghua | -- |
| Tunix |  |
2025.4 | -- | |
| RL2 |  |
2025.4 | Accio | – |
| verifiers |  |
2025.3 | Individual | -- |
| prime-rl |  |
2025.2 | Prime Intellect | -- |
| oat |  |
2024.11 | NUS/Sea AI | Paper |
| veRL |  |
2024.10 | ByteDance | Paper |
| OpenRLHF |  |
2023.7 | OpenRLHF | Paper |
| trl |  |
2019.11 | HuggingFace | -- |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| AgentJet | GRPO/PPO (swarm, multi-dim reward) | Both | Both | Multi | Swarm agentic RL (heterogeneous multi-agent, multi-task) | All (Custom/External/Rule) | Yes (tool calls, agent frameworks) |
| HarnessX | GRPO/PPO (slime/verl recipes) | Single | Outcome | Multi | Composable agent-harness foundry (ALFWorld/GAIA/WebShop/SWE-bench) | External + Custom | Yes (harness orchestrates tools/memory) |
| Polar | GRPO | Both | Outcome | Multi | Agentic RL on any harness (SWE-Bench/SWE-Gym) | External Verifier | Yes (real agent harnesses: shell/Codex/Claude Code) |
| uni-agent | GRPO/GSPO (partial rollout, fully-async) | Single | Outcome | Multi | SWE-Bench/Search/General Agent (1000+ concurrent) | All | Yes (unified model/tool/env abstractions) |
| VeRL-Omni | FlowGRPO/DanceGRPO/Diffusion DPO | Single | Outcome | Single | Multimodal generation RL (image/video/omni) | Model/External | No |
| OpenClaw-RL | GRPO/OPD | Both | Both | Multi | Terminal/GUI/SWE/Tool-call | Model/External | Yes |
| Claw-R1 | Generic RL Framework | Multi | Both | Multi | General Agent | All | Yes (Framework-agnostic) |
| Open-AgentRL | GRPO-TCR | Single | Both | Multi | Reasoning/GUI/Coding | Model (PRM) | Yes (SandboxFusion) |
| NeMo-RL | GRPO/DAPO/GDPO/DPO | Single | Outcome | Multi | Math/Reasoning/Code | Rule/External | No |
| RLinf | PPO/GRPO/DAPO/SAC/REINFORCE++/CrossQ/RLPD | Both | Both | Multi | Robotics/Math/Code/QA/VQA | All (Rule/Model/External) | Yes |
| siiRL | PPO/GRPO/CPGD/MARFT | Multi | Both | Multi | LLM/VLM/LLM-MAS PostTraining | Model/Rule | Planned |
| slime | GRPO/GSPO/REINFORCE++ | Single | Both | Both | Math/Code | External Verifier | Yes |
| agent-lightning | PPO/Custom/Automatic Prompt Optimization | Multi | Outcome | Multi | Calculator/SQL | Model/External/Rule | Yes |
| AReaL | PPO | Both | Outcome | Both | Math/Code | External | Yes |
| ROLL | PPO/GRPO/Reinforce++/TOPR/RAFT++ | Multi | Both | Multi | Math/QA/Code/Alignment | All | Yes |
| MARTI | PPO/GRPO/REINFORCE++/TTRL | Multi | Both | Multi | Math | All | Yes |
| Tunix | PPO/GRPO/GSPO-Token/DAPO/Dr.GRPO | Single | Outcome | Multi | Math/Code/Game | Rule/External | Yes |
| RL2 | Dr. GRPO/PPO/DPO | Single | Both | Both | QA/Dialogue | Rule/Model/External | Yes |
| verifiers | GRPO | Multi | Outcome | Both | Reasoning/Math/Code | All | Code |
| prime-rl | GRPO/PPO | Multi | Outcome | Multi | Math/Code/Search | Model/External | Yes |
| oat | PPO/GRPO | Single | Outcome | Multi | Math/Alignment | External | No |
| veRL | PPO/GRPO | Single | Outcome | Both | Math/QA/Reasoning/Search | All | Yes |
| OpenRLHF | PPO/REINFORCE++/GRPO/DPO/IPO/KTO/RLOO | Multi | Both | Both | Dialogue/Chat/Completion | Rule/Model/External | Yes |
| trl | PPO/GRPO/DPO | Single | Both | Single | QA | Custom | No |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| T2PO |  |
2026.5 | Academic (ICML 2026 Spotlight) | Paper | veRL |
| StraTA |  |
2026.5 | Shanghai AI Lab / Oxford / Multi-institution | Paper | rLLM |
| SDAR |  |
2026.5 | Zhejiang University (ZJU-REAL) | Paper | veRL (GiGPO-based) |
| SkillZero |  |
2026.4 | Zhejiang University (ZJU-REAL) | Paper | veRL (GiGPO-based) |
| MetaClaw |  |
2026.3 | UNC-Chapel Hill (AIMING Lab) | Paper | Custom |
| SkillRL |  |
2026.2 | UNC-Chapel Hill (AIMING Lab) | Paper | Custom |
| LLM-in-Sandbox |  |
2026.1 | RUC/MSRA/THU | Paper | rllm (w/ veRL) |
| youtu-agent |  |
2025.12 | Tencent Youtu Lab | Paper | Custom |
| DEPO |  |
2025.11 | HKUST/SJTU | Paper | LLaMA-Factory |
| SPEAR |  |
2025.10 | Tencent Youtu Lab | Paper | veRL/verl-agent |
| DeepAgent |  |
2025.10 | RUC/Xiaohongshu | Paper | Custom |
| AgentRL |  |
2025.9 | Tsinghua | Paper | veRL |
| AgentGym-RL |  |
2025.9 | Fudan University | Paper | veRL |
| Agent_Foundation_Models |  |
2025.8 | OPPO Personal AI Lab | Paper | veRL |
| Trinity-RFT |  |
2025.5 | Alibaba | Paper | veRL |
| SPA-RL-Agent |  |
2025.5 | PolyU | Paper | TRL |
| verl-agent |  |
2025.5 | NTU/Skywork | Paper | veRL |
| SkyRL |  |
2025.4 | UC Berkeley / NovaSky-AI | Paper | Self (skyrl-train) |
| VAGEN |  |
2025.3 | Northwestern University (mll-lab-nu) | Paper | veRL |
| ART |  |
2025.3 | OpenPipe | Paper | TRL |
| OpenManus-RL |  |
2025.3 | UIUC/MetaGPT | -- | Custom |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| T2PO | T²PO (token+turn uncertainty-guided) | Single | Both | Multi | WebShop/ALFWorld/SearchQA/Embody/Game | Rule | Yes (search, web, embodied) |
| StraTA | Hierarchical GRPO + Strategic Trajectory Abstraction | Single | Outcome | Multi | ALFWorld (93.1%)/WebShop (84.2%)/SciWorld (63.5%) | Rule + Model (self-judge) | Yes (interactive long-horizon envs) |
| SDAR | Self-Distilled Agentic RL (GRPO + gated OPSD) | Single | Outcome | Multi | ALFWorld/WebShop/Search-QA | Rule | Yes (interactive envs) |
| SkillZero | In-Context Agentic RL (GRPO + skill-context curriculum withdrawal) | Single | Outcome | Multi | ALFWorld/WebShop/Search-QA | Rule | Yes (interactive envs + skill library) |
| MetaClaw | GRPO (LoRA) | Single | Process | Multi | General Agentic | Model (PRM) | Yes (Skill-augmented) |
| SkillRL | GRPO | Single | Outcome | Multi | ALFWorld/WebShop/Search | Rule | Yes (Web search, actions) |
| LLM-in-Sandbox | GRPO++ | Single | Outcome | Multi | Math/Physics/Chemistry/Biomedicine/Long-context/IF/SWE | Rule | Yes (Code Sandbox w/ Terminal, File, Internet) |
| youtu-agent | Training-Free GRPO | Single | Outcome | Multi | Deep Research/Data Analysis/Tool-use | Model/External | Yes (Web search, code, file) |
| DEPO | KTO + Efficiency Loss | Single | Both | Multi | Agent (BabyAI/WebShop) | Rule | Yes |
| SPEAR | GRPO/GiGPO + SIL | Single | Both | Multi | Math/Agent | Rule/External | Yes (Search, Sandbox, Browser) |
| DeepAgent | ToolPO | Single | Outcome | Multi | ToolBench/ALFWorld/WebShop/GAIA/HLE | Model | Yes (16,000+ RapidAPIs) |
| AgentRL | GRPO/REINFORCE++/RLOO/ReMax/GAE | Single | Outcome | Multi | Agent Tasks | External | Yes |
| AgentGym-RL | PPO/GRPO/RLOO/REINFORCE++ | Single | Outcome | Multi | Web/Search/Game/Embodied/Science | Rule/Model/External | Yes (Web, Search, Env APIs) |
| Agent_Foundation_Models | DAPO/PPO | Single | Outcome | Single | QA/Code/Math | Rule/External | Yes |
| Trinity-RFT | PPO/GRPO | Single | Outcome | Both | Math/TextGame/Web | All | Yes |
| SPA-RL-Agent | PPO | Single | Process | Multi | Navigation/Web/TextGame | Model | No |
| verl-agent | PPO/GRPO/GiGPO/DAPO/RLOO/REINFORCE++ | Multi | Both | Multi | Phone Use/Math/Code/Web/TextGame | All | Yes |
| SkyRL | GRPO/PPO | Single | Both | Multi | Long-horizon Agents (SWE-Bench/Search/Math/SQL) | Rule/External/Custom | Yes |
| VAGEN | PPO/GRPO (World Modeling RL) | Single | Both | Multi | Navigation/TextGame/Multimodal | All | Yes |
| ART | GRPO | Multi | Both | Multi | TextGame | All | Yes |
| OpenManus-RL | PPO/DPO/GRPO | Multi | Outcome | Multi | TextGame | All | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| Harness-1 |  |
2026.6 | UIUC | Paper | Custom |
| SlimSearcher |  |
2026.6 | Ant Group / ZJU | Paper | Custom (agentic RL) |
| DeepRubric |  |
2026.6 | Shandong University | Paper | verl-tool |
| SAAS |  |
2026.5 | Xiamen University | Paper | slime |
| CuSearch |  |
2026.5 | Academic | Paper | Custom |
| ORBIT |  |
2026.4 | University of Waterloo | Paper | Custom |
| LiteResearcher |  |
2026.4 | Simplex AI / ZJU / PolyU | Paper | Custom |
| DR-Venus |  |
2026.4 | Ant Group (inclusionAI) | Paper | veRL (IGPO-based) |
| MR-Search |  |
2026.3 | Academic | Paper | Custom |
| ProRAG |  |
2026.1 | RUC | Paper | Custom |
| O-Researcher |  |
2026.1 | OPPO PersonalAI Lab | Paper | Custom |
| Agentic-RAG-R1 |  |
2025.12 | PKU | -- | Custom |
| MemSearcher |  |
2025.11 | CAS | Paper | Custom |
| DR Tulu |  |
2025.11 | AI2 / UW / CMU / MIT | Paper | Open-Instruct |
| IGPO |  |
2025.10 | Ant Group | Paper (ICLR 2026) | veRL |
| ReSeek |  |
2025.10 | Tencent PCG BAC/Tsinghua University | Paper | veRL |
| AutoGraph-R1 |  |
2025.10 | HKUST KnowComp | Paper | Custom |
| WebSeer |  |
2025.10 | Individual | Paper | veRL |
| HiPRAG |  |
2025.10 | Individual | Paper | veRL |
| Tree-GRPO |  |
2025.9 | AMAP | Paper | veRL |
| DeepResearch |  |
2025.9 | Alibaba/Tongyi Lab | Paper | Custom |
| DeepDive |  |
2025.9 | Tsinghua/THUDM | Paper | Custom |
| ASearcher |  |
2025.8 | Ant Research RL Lab Tsinghua University & UW |
Paper | RealHF/AReaL |
| SSRL |  |
2025.8 | Tsinghua | Paper | Custom |
| Research-Venus |  |
2025.8 | Ant Group | Paper | Custom |
| Graph-R1 |  |
2025.7 | BUPT/NTU/NUS | Paper | veRL |
| Kimi-Researcher |  |
2025.6 | Moonshot AI | blog | Custom |
| R-Search |  |
2025.6 | Individual | -- | veRL |
| R1-Searcher-plus |  |
2025.5 | RUC | Paper | Custom |
| StepSearch |  |
2025.5 | SenseTime | Paper | veRL |
| AutoRefine |  |
2025.5 | USTC | Paper | veRL |
| ZeroSearch |  |
2025.5 | Alibaba | Paper | veRL |
| ReasonRAG |  |
2025.5 | CityU HK / Huawei | Paper | Custom |
| VRAG |  |
2025.5 | USTC / Tongyi Lab, Alibaba | Paper | veRL |
| MaskSearch |  |
2025.5 | Tongyi Lab, Alibaba | Paper | DAPO / veRL |
| R3-RAG |  |
2025.5 | Fudan NLP | Paper | OpenRLHF |
| O2-Searcher |  |
2025.5 | KnowledgeXLab | Paper | veRL |
| s3 |  |
2025.5 | UIUC | Paper | veRL |
| knowledge-r1 |  |
2025.5 | CAS / UCAS | Paper | veRL |
| WebThinker |  |
2025.4 | RUC | Paper | Custom |
| DeepResearcher |  |
2025.4 | SJTU | Paper | veRL |
| Search-R1 |  |
2025.3 | UIUC/Google | paper1, paper2 | veRL |
| R1-Searcher |  |
2025.3 | RUC | Paper | OpenRLHF |
| C-3PO |  |
2025.2 | Alibaba | Paper | OpenRLHF |
| DeepRetrieval |  |
2025.2 | UIUC | Paper | veRL |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| Harness-1 | GRPO | Single | Outcome | Multi | Long-horizon search (web/finance/patents) w/ state-externalizing harness | External + Rule | Yes (search/retrieval/rerank) |
| SlimSearcher | GRPO + Adaptive Reward Gating | Single | Outcome | Multi | Efficiency-aware deep research (GAIA/BrowseComp/xBench) | Custom + Rule | Yes (web search, browse) |
| DeepRubric | GRPO + rubric rewards | Single | Process | Multi | Deep research report synthesis (evidence-tree rubric) | Model + Rule (rubric) | Yes (search/browse/scholar) |
| SAAS | RL w/ boundary-aware reward (2-stage curriculum) | Single | Outcome | Multi | Self-aware agentic search (over-search mitigation, 7 QA sets) | Rule-Based | Yes (search) |
| CuSearch | GRPO + Search-Depth curriculum rollout | Single | Outcome | Multi | Agentic RAG multi-hop QA | Rule-Based (EM) | Yes (retrieval/search) |
| ORBIT | GRPO | Single | Outcome | Multi | Verifiable data-gen + RL for web search (Qwen3-4B) | External + Rule | Yes (web search) |
| LiteResearcher | Scalable Agentic RL (curriculum w/ lite virtual world) | Single | Outcome | Multi | Deep Research (GAIA 71.3% / Xbench-DS 78.0%, 4B SOTA) | Rule/External | Yes (local search/browse env, Milvus+PostgreSQL) |
| DR-Venus | GRPO + IGPO (info-gain turn-level) w/ agentic SFT | Single | Both | Multi | Edge-scale Deep Research (4B) | Intrinsic (info-gain) + Rule (format) | Yes (Search/Browse) |
| MR-Search | In-context Meta-RL (multi-episode credit) | Single | Outcome | Multi | Agentic search w/ self-reflection | Rule-Based | Yes (search) |
| ProRAG | GRPO + DGA (dual-granularity advantage) | Single | Both | Multi | Multi-hop RAG | Model (PRM via MCTS) | Yes (Retrieval) |
| O-Researcher | GRPO + RLAIF | Multi | Process | Multi | Deep Research (Zhihu-KOL/WideSearch/ELI5) | Model (LLM-as-Judge) | Yes (Search/Crawl) |
| Agentic-RAG-R1 | GRPO | Single | Outcome | Multi | Knowledge-intensive QA | Rule/Model | Yes (Wiki/Doc search) |
| MemSearcher | Multi-context GRPO | Single | Outcome | Multi | Search/QA + Memory | Rule/Model | Yes (Web search + Memory) |
| DR Tulu | GRPO + evolving rubrics | Single | Outcome | Multi | Long-form Deep Research | Model (rubrics) | Yes (Search/MCP) |
| IGPO | GRPO + IGPO (Information Gain turn-level reward) | Single | Both | Multi | Multi-turn Search Agent (BrowseComp/-ZH) | Intrinsic (belief Δ) + Outcome | Yes (Search) |
| ReSeek | GRPO/PPO | Single | Both | Multi | QA/Search | Rule | Search/JUDGE |
| AutoGraph-R1 | GRPO (via VeRL) | Single | Outcome | Multi | KG Construction for QA | Rule | Yes (Graph retrieval) |
| WebSeer | GRPO-style | Single | Outcome | Multi | Web Search QA (w/ self-reflection) | Rule/Model | Yes (Search) |
| HiPRAG | PPO | Single | Process | Multi | Efficient Agentic RAG | Model/Rule | Yes (Retrieval) |
| Tree-GRPO | GRPO/Tree-GRPO | Single | Outcome | Multi | Search | Rule | Search |
| DeepResearch | RL-based | Single | Outcome | Multi | Deep Research | Model | Yes (Search, Browse) |
| DeepDive | GRPO | Single | Outcome | Multi | KG-augmented Search | Rule | Yes (KG + Search) |
| ASearcher | PPO/GRPO + Decoupled PPO | Single | Outcome | Multi | Math/Code/SearchQA | External/Rule | Yes |
| SSRL | GRPO | Single | Outcome | Multi | Self-Search | Rule | Yes (Self-search) |
| Research-Venus | GRPO | Single | Both | Multi | Deep Research | Model (atomic thought) | Yes (Search) |
| Graph-R1 | GRPO/REINFORCE++/PPO | Single | Outcome | Multi | KGQA | Rule (EM/F1) | Yes (Graph retrieval) |
| Kimi-Researcher | REINFORCE | Single | Outcome | Multi | Research | Outcome | Search, Browse, Coding |
| R-Search | PPO/GRPO | Single | Both | Multi | QA/Search | All | Yes |
| R1-Searcher-plus | Custom | Single | Outcome | Multi | Search | Model | Search |
| StepSearch | PPO | Single | Process | Multi | QA | Model | Search |
| AutoRefine | PPO/GRPO | Multi | Both | Multi | RAG QA | Rule | Search |
| ZeroSearch | PPO/GRPO/REINFORCE | Single | Outcome | Multi | QA/Search | Rule | Yes |
| ReasonRAG | DPO + MCTS-based PRM | Single | Process | Multi | Multi-hop QA | Model (PRM) | Yes (Wikipedia search) |
| VRAG | GRPO | Single | Both | Multi | Visually-rich RAG | Rule/Model | Yes (Visual retrieval) |
| MaskSearch | DAPO | Single | Outcome | Multi | RAMP Pretraining + QA | Rule/Model | Yes (Search) |
| R3-RAG | PPO | Single | Both | Multi | Multi-hop QA | Rule | Yes (Retrieval) |
| O2-Searcher | GRPO | Single | Outcome | Multi | Open-ended QA | Rule/Model | Yes (Search) |
| s3 | GRPO | Single | Outcome | Multi | RAG / Medical QA | Model (Gain-Beyond-RAG) | Yes (Retrieval) |
| knowledge-r1 | GRPO | Single | Outcome | Multi | Knowledge-intensive QA (KB-aware) | Rule | Yes (Retrieval) |
| WebThinker | DPO | Single | Outcome | Multi | Reasoning/QA/Research | Model/External | Web Browsing |
| DeepResearcher | PPO/GRPO | Multi | Outcome | Multi | Research | All | Yes |
| Search-R1 | PPO/GRPO | Single | Outcome | Multi | Search | All | Search |
| R1-Searcher | PPO/DPO | Single | Both | Multi | Search | All | Yes |
| C-3PO | PPO | Multi | Outcome | Multi | Search | Model | Yes |
| DeepRetrieval | GRPO | Single | Outcome | Multi | Query Generation/IR | Rule | Yes (Search) |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| SCALE-CUA |  |
2026.7 | Tsinghua (THUDM) | Paper | Custom (Ray + vLLM + Megatron-LM) |
| OpenWebRL |  |
2026.6 | UIUC / Microsoft Research | Paper | slime |
| ToolCUA |  |
2026.5 | Alibaba Tongyi Lab (X-PLUG) | Paper | Custom |
| ClawGUI |  |
2026.4 | Zhejiang University (ZJU-REAL) | Paper | Custom (veRL-based) |
| OpAgent |  |
2026.2 | Codefuse AI (Ant Group) | Paper | Agent-R1 (veRL) |
| GUI-Libra |  |
2026.2 | GUI-Libra (MS-affiliated) | Paper | Custom |
| MobileAgent |  |
2025.9 | X-PLUG (TongyiQwen) | paper | veRL |
| UI-TARS |  |
2025.9 | ByteDance Seed | Paper | Custom |
| MobileRL |  |
2025.9 | Tsinghua / Zhipu AI (THUDM) | Paper | Custom |
| DART-GUI |  |
2025.9 | Computer-use-agents | Paper | veRL |
| Mano-P |  |
2025.9 | Mininglamp AI | Paper | Mano-SDK |
| InfiGUI-G1 |  |
2025.8 | InfiX AI | Paper | veRL |
| gui-rcpo |  |
2025.8 | Zhejiang University | Paper | Custom |
| UI-AGILE |  |
2025.7 | Xiamen University | Paper | Custom |
| GUI-G2 |  |
2025.7 | Zhejiang University (ZJU-REAL) | Paper | Custom (VLM-R1) |
| MagicGUI |  |
2025.7 | Honor (MagicAgent-GUI) | Paper | Custom |
| Grounding-R1 |  |
2025.6 | Salesforce | blog | trl |
| AgentCPM-GUI |  |
2025.6 | OpenBMB/Tsinghua/RUC | Paper | Huggingface |
| TTI |  |
2025.6 | CMU | Paper | Custom |
| GTA1 |  |
2025.6 | Salesforce / ANU | Paper | Custom (DeepSpeed) |
| SE-GUI |  |
2025.5 | Nankai University/vivo | Paper | trl |
| ARPO |  |
2025.5 | CUHK/HKUST | Paper | veRL |
| GUI-G1 |  |
2025.5 | RUC | Paper | TRL |
| WebAgent-R1 |  |
2025.5 | Amazon/UVA | Paper | Custom |
| ZeroGUI |  |
2025.5 | Shanghai AI Lab | Paper | Custom |
| GUI-R1 |  |
2025.4 | CAS/NUS | Paper | veRL |
| InfiGUI-R1 |  |
2025.4 | Zhejiang University | Paper | Custom |
| UI-R1 |  |
2025.3 | vivo/CUHK | Paper | TRL |
| CollabUIAgents |  |
2025.2 | Tsinghua/Alibaba/HKUST | Paper | Custom |
| DigiQ |  |
2025.2 | UC Berkeley/CMU/Amazon | Paper | Custom |
| GUI-Agent-RL |  |
2025.2 | Microsoft | Paper | Custom |
| WebAgent |  |
2025.1 | Alibaba | paper1, paper2 | LLaMA-Factory |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| SCALE-CUA | GRPO (fully async, Frontier Sampling + visual context segmentation) | Single | Outcome | Multi | Computer Use (OSWorld, ScienceBoard; 24K+ synthesized verifiable tasks) | External Verifier (executable judge functions) | Yes (GUI actions in Docker desktop) |
| OpenWebRL | GRPO (online multi-turn) | Single | Both | Multi | Visual web browsing on live sites (WebVoyager/Online-Mind2Web) | Rule + Model (format + LLM-judge) | Yes (Playwright browser) |
| ToolCUA | Tool-Bootstrapped GUI RFT + Online Agentic RL (Tool-Efficient Path Reward) | Single | Both | Multi | Computer Use (OSWorld-MCP, hybrid GUI+tool) | Rule (path-efficiency) | Yes (GUI actions + structured tool calls) |
| ClawGUI | GiGPO + Process Reward Model | Single | Both | Multi | Mobile GUI (Android/HarmonyOS/iOS, MobileWorld) | Rule + Model (PRM) | Yes (GUI + hybrid CLI-GUI + persistent memory) |
| OpAgent | Online agentic RL (GRPO/PPO) | Multi | Both | Multi | Web navigation (WebArena 71.6% pass@5) | Rule + Model (RDTree + WebJudge) | Yes (Playwright browser) |
| GUI-Libra | KL-regularized GRPO (Partially Verifiable RL) | Single | Outcome | Multi | GUI (AndroidWorld/WebArena/Online-Mind2Web) | Rule | Yes |
| MobileAgent | semi-online RL | Single | Both | Multi | MobileGUI/Automation | Rule | Yes |
| UI-TARS | Multi-turn RL | Single | Both | Multi | GUI (Cross-platform) | Model | Yes (GUI actions) |
| MobileRL | AdaGRPO (Difficulty-Adaptive) | Single | Outcome | Multi | Mobile GUI (AndroidWorld/AndroidLab) | Rule | Yes (Android) |
| DART-GUI | Decoupled GRPO | Single | Outcome | Multi | GUI (OSWorld) | Rule | Yes |
| Mano-P | Three-stage SFT→Offline RL→Online RL | Single | Both | Multi | GUI (OSWorld) | Rule | Yes |
| InfiGUI-G1 | AEPO | Single | Outcome | Single | GUI/Grounding | Rule | No |
| gui-rcpo | RCPO | Single | Outcome | Single | GUI Grounding | Rule (self-supervised) | No |
| UI-AGILE | GRPO | Single | Outcome | Single | GUI Grounding | Rule (continuous) | No |
| GUI-G2 | GRPO (Gaussian Reward) | Single | Outcome | Single | GUI Grounding | Rule (continuous) | No |
| MagicGUI | Reinforcement Fine-Tuning (RFT) | Single | Outcome | Multi | Mobile GUI | Model/Rule | Yes |
| Grounding-R1 | GRPO | Single | Outcome | Multi | GUI Grounding | Model | Yes |
| AgentCPM-GUI | GRPO | Single | Outcome | Multi | Mobile GUI | Model | Yes |
| TTI | REINFORCE/BC | Single | Outcome | Multi | Web | External | Web Browsing |
| GTA1 | GRPO-style (click-success reward) | Single | Outcome | Multi | GUI Grounding (OSWorld/ScreenSpot-Pro) | Rule | Yes |
| SE-GUI | GRPO | Single | Both | Single | GUI Grounding | Rule | Yes |
| ARPO | GRPO | Single | Outcome | Multi | GUI | External | Computer Use |
| GUI-G1 | GRPO | Single | Outcome | Single | GUI | Rule/External | No |
| WebAgent-R1 | M-GRPO | Single | Outcome | Multi | Web Navigation (WebArena-Lite) | Rule (task success) | Yes (Web browsing) |
| ZeroGUI | Online RL | Single | Outcome | Multi | GUI Agent | Rule | Yes (GUI actions) |
| GUI-R1 | GRPO | Single | Outcome | Multi | GUI | Rule | No |
| InfiGUI-R1 | RL + sub-goal guidance | Single | Both | Multi | GUI Reasoning | Rule | Yes |
| UI-R1 | GRPO | Single | Process | Both | GUI | Rule | Computer/Phone Use |
| CollabUIAgents | DPO (credit re-assignment) | Multi | Process | Multi | GUI (Mobile + Web) | Model (LLM) | Yes (GUI interaction) |
| DigiQ | Value-based offline RL | Single | Outcome | Multi | Android Device Control | Model (Q-function) | Yes |
| GUI-Agent-RL | Value-based RL (VEM) | Single | Outcome | Multi | GUI (Web Shopping) | Model | Yes |
| WebAgent | DAPO | Multi | Process | Multi | Web | Model | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| Tool-RL-Box |  |
2026.6 | Harbin Institute of Technology | Paper | veRL (w/ verl-tool) |
| SPADER |  |
2026.6 | Zhejiang University | Paper | veRL |
| APPO |  |
2026.6 | Alibaba AMAP (AMAP-ML) | Paper | veRL |
| AgenticQwen |  |
2026.4 | Alibaba PAI | Paper | veRL (w/ EasyDistill) |
| Agent-STAR |  |
2026.3 | CUHK | Paper | veRL |
| ToolOrchestra |  |
2025.11 | NVIDIA / HKU | Paper | Custom (veRL-based) |
| ToolMaster |  |
2025.11 | Northeastern University (NEUIR) | Paper | Custom |
| MATPO |  |
2025.10 | MiroMind AI | Paper | Custom |
| AWorld-RL |  |
2025.10 | Ant Group (inclusionAI) | -- | AWorld + veRL |
| CodeGym |  |
2025.9 | Academic | Paper | Custom |
| UserRL |  |
2025.9 | Salesforce AI Research | Paper | veRL |
| ToolBrain |  |
2025.9 | ToolBrain (AAMAS 2026) | Paper | Custom |
| Tool-R1 |  |
2025.9 | Individual (YBYBZhang) | Paper | Custom |
| MiroRL |  |
2025.8 | MiroMindAI | HF Repo | veRL |
| verl-tool |  |
2025.6 | TIGER-Lab | X | veRL |
| Multi-Turn-RL-Agent |  |
2025.5 | University of Minnesota | Paper | Custom |
| Tool-N1 |  |
2025.5 | NVIDIA | Paper | veRL |
| Tool-Star |  |
2025.5 | RUC | Paper | LLaMA-Factory |
| RL-Factory |  |
2025.5 | Simple-Efficient | model | veRL |
| calculator_agent_rl |  |
2025.5 | Individual (Danau5tin) | -- | Verifiers |
| ReTool |  |
2025.4 | ByteDance | Paper | veRL |
| ToolRL |  |
2025.4 | UIUC | Paper | veRL |
| AWorld |  |
2025.3 | Ant Group (inclusionAI) | Paper | veRL |
| Agent-R1 |  |
2025.3 | USTC | Paper | veRL |
| ReCall |  |
2025.3 | BaiChuan | Paper | veRL |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| Tool-RL-Box | GRPO + supervisory signals (anti format-collapse) | Single | Process | Multi | Multi-step function calling (FCL / ToolACE, pluggable tool servers) | Model (LLM-judge error taxonomy) + Rule | Yes (function-calling tools) |
| SPADER | GRPO + Step-wise Peer Advantage (SPA) | Single | Both | Multi | Long-horizon tool-augmented multi-answer QA (QAMPARI) | Rule-Based (entity-match + diversity) | Yes (search) |
| APPO | APPO (procedure-aware branching; extends ARPO/GRPO) | Single | Process | Multi | Multi-turn TIR (reasoning+search+code, 13 benchmarks) | Rule-Based | Yes (search + code) |
| AgenticQwen | Multi-round RL (Reasoning RL + Agentic RL w/ dual data flywheels) | Single | Outcome | Multi | Industrial Tool Use (search, data analysis, tau-bench airline/retail/telecom) | Rule + Model (rubric) | Yes (Python interpreter, web search, mock tools) |
| Agent-STAR | GRPO + dense/curriculum reward (STAR recipe) | Single | Both | Multi | Long-horizon tool-using agents (TravelPlanner, ReAct up to 60 turns) | Rule + External | Yes (planning APIs) |
| ToolOrchestra | End-to-end RL (outcome+efficiency+preference) | Single | Both | Multi | Tool orchestration / agentic workflows | All | Yes (Search/Code/LLMs) |
| ToolMaster | SFT + GRPO (trial-then-execute) | Single | Outcome | Multi | Tool trialing + execution (ToolHop/TMDB/StableToolBench) | Rule/External | Yes (Simulated tools) |
| MATPO | GRPO (multi-agent) | Multi | Outcome | Multi | Tool-use/Search | Rule | Yes (MCP: Serper, Web scraping) |
| AWorld-RL | Collection: RODS / HardGen / FunReason-MT / Environment Tuning / V2P / RAG-R1 | Both | Both | Multi | Multi-turn function calling + GUI grounding + deep search (BFCL etc.) | Rule + Model (progress reward) | Yes (function calls, GUI, search) |
| CodeGym | GRPO-family | Single | Outcome | Multi | Synthetic Multi-turn Tool-Use | Rule (verifiable) | Yes (Synthesized tools) |
| UserRL | GRPO (multi-turn credit) | Single | Both | Multi | User-centric (Function/Persuade/Search/Tau Gyms) | Model/External | Yes |
| ToolBrain | GRPO/DPO | Single | Outcome | Multi | Agentic tool training | Rule/Model | Yes (User-defined tools) |
| Tool-R1 | Policy optimization (PPO-style) | Single | Outcome | Multi | Agentic Tool Use (GAIA) | Model + External | Yes (Python exec) |
| MiroRL | GRPO | Single | Both | Multi | Reasoning/Planning/ToolUse | Rule-based | MCP |
| verl-tool | PPO/GRPO | Single | Both | Both | Math/Code | Rule/External | Yes |
| Multi-Turn-RL-Agent | GRPO | Single | Both | Multi | Tool-use/Math | Rule/External | Yes |
| Tool-N1 | PPO | Single | Outcome | Multi | Math/Dialogue | All | Yes |
| Tool-Star | PPO/DPO/ORPO/SimPO/KTO | Single | Outcome | Multi | Multi-modal/Tool Use/Dialogue | Model/External | Yes |
| RL-Factory | GRPO | Multi | Both | Multi | Tool-use/NL2SQL | All | MCP |
| calculator_agent_rl | GRPO | Single | Outcome | Multi | Calculator Tool Use | Model (Claude-judge) | Yes |
| ReTool | PPO | Single | Outcome | Multi | Math | External | Code |
| ToolRL | GRPO/PPO | Single | Outcome | Multi | Tool Learning | Rule/External | Yes |
| AWorld | GRPO | Both | Outcome | Multi | Search/Web/Code | External/Rule | Yes |
| Agent-R1 | PPO/GRPO | Single | Both | Multi | Tool-use/QA | Model | Yes |
| ReCall | PPO/GRPO/RLOO/REINFORCE++/ReMax | Single | Outcome | Multi | Tool-use/Math/QA | All | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| FastContext |  |
2026.6 | Microsoft | Paper | Custom |
| SWE-Edit |  |
2026.4 | Microsoft Research | Paper | Custom |
| CodeScout |  |
2026.3 | OpenHands | Paper | SkyRL |
| CUDA-Agent |  |
2026.2 | ByteDance/Tsinghua | Paper | Custom |
| SWE-World |  |
2026.2 | RUC (RUCAIBox) | Paper | OpenRLHF + veRL |
| LLM-in-Sandbox | |
2026.1 | RUC/MSRA/THU | Paper | rllm (w/ veRL) |
| CUDA-L2 |  |
2026.1 | DeepReinforce AI | Paper | Custom |
| PPP-Agent |  |
2025.11 | CMU/OpenHands | Paper | veRL |
| DeepAnalyze |  |
2025.10 | RUC/Tsinghua | Paper | Custom |
| RepoDeepSearch |  |
2025.8 | PKU, Bytedance, BIT | Paper | veRL |
| CUDA-L1 |  |
2025.7 | DeepReinforce AI | Paper | Custom |
| SWE-Swiss |  |
2025.7 | Tsinghua / ByteDance | -- | veRL |
| MedAgentGym |  |
2025.6 | Emory/Georgia Tech | Paper | Hugginface |
| CURE |  |
2025.6 | University of Chicago Princeton/ByteDance |
Paper | Huggingface |
| Time-R1 |  |
2025.5 | UIUC | Paper | veRL |
| ML-Agent |  |
2025.5 | MASWorks | Paper | Custom |
| R1-Code-Interpreter |  |
2025.5 | MIT | Paper | Custom |
| digitalhuman |  |
2025.4 | Tencent | Paper | veRL |
| Skywork-OR1 |  |
2025.4 | Skywork AI | Paper | Custom (veRL fork) |
| sweet_rl |  |
2025.3 | Meta/UCB | Paper | OpenRLHF |
| swe-rl |  |
2025.2 | Meta/UIUC/CMU | Paper | Custom |
| CTRL |  |
2025.2 | HKU/ByteDance | Paper | Custom |
| AceCoder |  |
2025.2 | Waterloo (TIGER-Lab) | Paper | Custom |
| rllm |  |
2025.1 | Berkeley Sky Computing Lab BAIR / Together AI |
Notion Blog | veRL |
| open-r1 |  |
2025.1 | HuggingFace | -- | TRL |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| FastContext | Task-grounded RL | Single | Outcome | Multi | Repo-explorer subagent (context gathering + citations) | Rule-Based | Yes (Read/Glob/Grep, parallel) |
| SWE-Edit | GRPO (adaptive mode selection) | Multi (Viewer + Editor subagents) | Outcome | Multi | SWE-bench Verified (find-replace vs whole-file rewrite) | Rule/External (test-based) | Yes (bash, file ops, viewer subagent) |
| CodeScout | GSPO | Single | Outcome | Multi | Repo-level code search/localization (terminal) | Rule-Based (F1) | Yes (terminal: rg/sed/cat) |
| CUDA-Agent | Agentic RL (staged) | Single | Outcome | Multi | CUDA Kernel Generation | Rule (correctness + performance) | Yes (compile/verify/profile) |
| SWE-World | RL with learned world model (SWT + SWR) | Single | Both | Multi | Docker-free SWE (SWE-Bench Verified) | Model (surrogate) + Rule | Yes |
| LLM-in-Sandbox | GRPO++ | Single | Outcome | Multi | Code/SWE + General (Math/Sci/Bio) | Rule | Yes (Code Sandbox w/ Terminal, File, Internet) |
| CUDA-L2 | Contrastive RL | Single | Outcome | Single | HGEMM / CUDA Matmul | Rule (TFLOPs) | Yes (compile/benchmark) |
| PPP-Agent | PPP-RL | Single | Both | Multi | SWE/Research | Rule+Model | Search, Ask, Browse |
| DeepAnalyze | Curriculum RL | Single | Outcome | Multi | Data Science | Rule/External | Yes (Code exec) |
| RepoDeepSearch | GRPO | Single | Both | Multi | Search/Repair | Rule/External | Yes |
| CUDA-L1 | Contrastive RL | Single | Outcome | Single | CUDA Optimization | Rule (performance) | No |
| SWE-Swiss | Two-stage RL curriculum | Single | Outcome | Multi | SWE (Localization/Repair/Unit-Test) | Rule (test-based) | Yes |
| MedAgentGym | SFT/DPO/PPO/GRPO | Single | Outcome | Multi | Medical/Code | External | Yes |
| CURE | PPO | Single | Outcome | Single | Code | External | No |
| Time-R1 | PPO/GRPO/DPO | Multi | Outcome | Multi | Temporal | All | Code |
| ML-Agent | Custom | Single | Process | Multi | Code | All | Yes |
| R1-Code-Interpreter | GRPO | Single | Outcome | Multi | Code Interpretation | Rule/External | Yes (Code exec) |
| digitalhuman | PPO/GRPO/ReMax/RLOO | Multi | Outcome | Multi | Empathy/Math/Code/MultimodalQA | Rule/Model/External | Yes |
| Skywork-OR1 | Large-scale rule-based RL (GRPO variant) | Single | Outcome | Single | Math + Code (AIME/LiveCodeBench) | Rule (verifiable) | No |
| sweet_rl | DPO | Multi | Process | Multi | Design/Code | Model | Web Browsing |
| swe-rl | RL-based | Single | Outcome | Single | SWE (SWE-bench) | Rule (similarity) | No |
| CTRL | RL (critique-revision) | Single | Process | Multi | Code Refinement | Model | Yes (Code exec) |
| AceCoder | GRPO | Single | Outcome | Single | Code Generation | External (test cases) | Yes |
| rllm | PPO/GRPO | Single | Outcome | Multi | Code Edit | External | Yes |
| open-r1 | GRPO | Single | Outcome | Single | Math/Code | All | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| Agent0 |  |
2025.10 | UNC‑Chapel Hill / Salesforce Research / Stanford University | Paper | veRL |
| KG-R1 |  |
2025.9 | UIUC/Google | Paper1, Paper2 | veRL |
| AgentFlow |  |
2025.09 | Stanford University | arXiv | veRL |
| THOR |  |
2025.9 | USTC / iFLYTEK | Paper | veRL |
| Tool-Light |  |
2025.9 | RUC (RUC-NLPIR) | Paper | LLaMA-Factory |
| ARPO |  |
2025.7 | RUC, Kuaishou | Paper | veRL |
| terminal-bench-rl |  |
2025.7 | Individual (Danau5tin) | N/A | rLLM |
| AutoTIR |  |
2025.7 | Beihang University / BAAI | Paper | veRL |
| MOTIF |  |
2025.6 | University of Maryland | Paper | trl |
| cmriat/l0 |  |
2025.6 | CMRIAT | Paper | veRL |
| agent-distillation |  |
2025.5 | KAIST | Paper | Custom |
| EasyR1 |  |
2025.4 | Individual | repo1/paper2 | veRL |
| AutoCoA |  |
2025.3 | BJTU | Paper | veRL |
| ToRL |  |
2025.3 | SJTU | Paper | veRL |
| ReMA |  |
2025.3 | SJTU, UCL | Paper | veRL |
| Agentic-Reasoning |  |
2025.2 | Oxford | Paper | Custom |
| SimpleTIR |  |
2025.2 | NTU, Bytedance | Notion Blog | veRL |
| openrlhf_async_pipline |  |
2024.5 | OpenRLHF | Paper | OpenRLHF |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| Agent0 | ADPO | Multi | Process | Multi | Math/Visual | Model/Verifier | Yes |
| KG-R1 | GRPO/PPO | Single | Both | Multi | KGQA | Rule/Model | KG Retrieval |
| AgentFlow | Flow-GRPO | Single | Outcome | Multi | Search/Math/QA | Model/External | Yes |
| THOR | Hierarchical GRPO (trajectory+step) | Single | Both | Multi | Math (MATH500/AIME/Olympiad) | External (SandboxFusion) | Yes (Python) |
| Tool-Light | Self-Evolved DPO | Single | Outcome | Multi | Tool-Integrated Reasoning | Model (preference) | Yes (FlashRAG/Python) |
| ARPO | GRPO | Single | Outcome | Multi | Math/Coding | Model/Rule | Yes |
| terminal-bench-rl | GRPO | Single | Outcome | Multi | Coding/Terminal | Model+External Verifier | Yes |
| AutoTIR | PPO | Single | Outcome | Multi | Autonomous Tool Selection (QA/Math/IF) | Rule | Yes (Search/Python) |
| MOTIF | GRPO | Single | Outcome | Multi | QA | Rule | No |
| cmriat/l0 | PPO | Multi | Process | Multi | QA | All | Yes |
| agent-distillation | PPO | Single | Process | Multi | QA/Math | External | Yes |
| EasyR1 | GRPO | Single | Process | Multi | Vision-Language | Model | Yes |
| AutoCoA | GRPO | Multi | Outcome | Multi | Reasoning/Math/QA | All | Yes |
| ToRL | GRPO | Single | Outcome | Single | Math | Rule/External | Yes |
| ReMA | PPO | Multi | Outcome | Multi | Math | Rule | No |
| Agentic-Reasoning | Custom | Single | Process | Multi | QA/Math | External | Web Browsing |
| SimpleTIR | PPO/GRPO (with extensions) | Single | Outcome | Multi | Math, Coding | All | Yes |
| openrlhf_async_pipline | PPO/REINFORCE++/DPO/RLOO | Single | Outcome | Multi | Dialogue/Reasoning/QA | All | No |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| Maestro |  |
2026.5 | Tsinghua / Multi-institution | Paper | veRL + verl-tool |
| DrMAS |  |
2026.2 | NTU | Paper | Custom |
| MarsRL |  |
2025.11 | Academic | Paper | veRL |
| PettingLLMs |  |
2025.10 | Intel / UCSD | Paper | Custom |
| MASPRM |  |
2025.10 | UBC / Huawei | Paper | Custom |
| MrlX |  |
2025.10 | Ant Group (AQ-MedAI) | Paper | Custom (SGLang + Megatron) |
| CoMAS |  |
2025.10 | Shanghai AI Lab / CUHK / Oxford / NUS | Paper | Custom |
| MAPoRL |  |
2025.8 | Academic | -- | Custom |
| CoMLRL |  |
2025.8 | OpenMLRL | Paper | TRL |
| ARIA |  |
2025.6 | Fudan University | Paper | Custom |
| SPIRAL |  |
2025.6 | NUS / A*STAR / Sea AI Lab | Paper | Oat |
| AMPO |  |
2025.5 | Tongyi Lab, Alibaba | Paper | veRL |
| FlowReasoner |  |
2025.4 | Sea AI Lab / NUS | Paper | Custom |
| MARFT |  |
2025.4 | SII / SJTU | Paper | Custom |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| Maestro | Outcome RL (lightweight orchestrator over frozen expert ensembles) | Multi (orchestrator + frozen experts) | Outcome | Multi | 10 multimodal benchmarks (math/chart/HR/domain — 70.1% avg, beats GPT-5 & Gemini-2.5-Pro) | External | Yes (expert models + 2-tier skill library: OCR/detection/visual) |
| DrMAS | GRPO (agent-wise) | Multi | Outcome | Multi | Multi-agent LLM Systems | Rule | No |
| MarsRL | RLVR (agent-specific rewards) | Multi | Both | Multi | Math Reasoning (AIME/BeyondAIME) | Rule (verifiable) | No |
| PettingLLMs | AT-GRPO | Multi | Both | Multi | Game/Code/Math/Planning | Rule (verifiable) | No |
| MASPRM | PRM (trained from MCTS rollouts) | Multi | Process | Multi | Reasoning (GSM8K/MATH/MMLU) | Learned PRM | No |
| MrlX | M-GRPO (hierarchical) | Multi | Outcome | Multi | Deep Research (GAIA/XBench) | Rule + Model | Yes (Search) |
| CoMAS | RL w/ LLM-Judge intrinsic reward | Multi | Process | Multi | Co-evolving Reasoning | Model | No |
| MAPoRL | PPO | Multi | Outcome | Multi | Collaborative LLM Tasks | Rule | No |
| CoMLRL | MAGRPO / MAREINFORCE / MARLOO | Multi | Outcome | Multi | Writing / Code / Minecraft | Custom | Minimal |
| ARIA | REINFORCE | Both | Process | Multi | Negotiation/Bargaining | Other | No |
| SPIRAL | Role-conditioned Advantage Estimation (RAE) | Multi | Outcome | Multi | Zero-sum Games (TicTacToe/Kuhn/Negotiation) | Rule | No |
| AMPO | BC/AMPO(GRPO improvement) | Multi | Outcome | Multi | Social Interaction | Model-based | No |
| FlowReasoner | GRPO | Multi | Outcome | Multi | Multi-agent Workflow Design | Rule | Yes |
| MARFT | MARFT paradigm (action+token level) | Multi | Both | Multi | Research / Math | Rule | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| Supersede |  |
2026.6 | Vrin | Paper | verifiers + prime-rl |
| AgeMem |  |
2026.4 | Multi-institution (incl. Alibaba DAMO) | Paper | Trinity-RFT |
| Mem-alpha |  |
2025.9 | UCSD / USTC | Paper | veRL |
| MEM1 |  |
2025.7 | MIT | Paper | veRL (based on Search-R1) |
| M3-Agent |  |
2025.7 | ByteDance Seed / Zhejiang University | Paper | Custom |
| Memento |  |
2025.6 | UCL, Huawei | Paper | Custom |
| MemAgent |  |
2025.6 | Bytedance, Tsinghua-SIA | Paper | veRL |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| Supersede | GRPO (+ LoRA) | Single | Outcome | Multi | Memory-update gap: keeping notes current across sessions (LongMemEval knowledge-update) | Rule-Based (answered_current / stale_penalty) | Yes (capped notes memory as action space) |
| AgeMem | Step-wise GRPO (3-stage progressive RL) | Single | Process | Multi | Unified LTM/STM management (memory ops as tools) | Rule (task accuracy + memory quality) | Yes (store/retrieve/update/summarize/discard memory tools) |
| Mem-alpha | GRPO | Single | Outcome | Multi | Long-context QA + Memory Construction | Rule (downstream QA) | Yes (memory tools) |
| MEM1 | PPO/GRPO | Single | Outcome | Multi | WebShop/GSM8K/QA | Rule/Model | Yes |
| M3-Agent | RL-based | Single | Outcome | Multi | Long-video QA (M3-Bench) | Rule/Model | Yes (multimodal memory graph) |
| Memento | soft Q-Learning | Single | Outcome | Multi | Research/QA/Code/Web | External/Rule | Yes |
| MemAgent | PPO, GRPO, DPO | Multi | Outcome | Multi | Long-context QA | Rule/Model/External | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| REAL |  |
2026.7 | InternRobotics | Paper | Custom (GSPO/GRPO over MCP) |
| Embodied-R1.5 |  |
2026.6 | Tianjin University | Paper | EasyR1 / veRL |
| AVA-VLA |  |
2026.6 | UCAS | Paper | Custom (PPO) |
| WorldVLN |  |
2026.5 | Tsinghua (EmbodiedCity) | Paper | Custom |
| Embodied-R1 |  |
2025.6 | Tianjing University | Paper | veRL |
| VIKI-R |  |
2025.6 | MARS-EAI (NeurIPS 2025 D&B) | Paper | veRL + LLaMA-Factory |
| STeCa |  |
2025.2 | The Hong Kong Polytechnic University | Paper | FastChat/TRL |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| REAL | GRPO/GSPO (online RL over an MCP tool interface) | Single | Outcome | Multi | Open-world mobile manipulation in Isaac Sim (REAL-Bench, 241 tasks) | External Verifier (target world-state check) | Yes (8 MCP tools: navigate_to/pick/place/ask/...) |
| Embodied-R1.5 | RFT (GRPO-family multimodal) | Single | Outcome | Multi | Embodied foundation model w/ Planner-Grounder-Corrector closed-loop | Rule-Based | No (closed-loop PGC) |
| AVA-VLA | PPO (latent reasoning as sequential decision) | Single | Both | Multi | VLA manipulation (LIBERO/ALOHA), latent CoT w/ early-exit | External (task success) + Custom | No (closed-loop manipulation) |
| WorldVLN | Action-aware GRPO | Single | Both | Multi | Aerial (UAV) vision-language navigation (closed-loop) | Rule + Model | No (closed-loop UAV control) |
| Embodied-R1 | GRPO | Single | Outcome | Single | Grounding/Waypoint | Rule | No |
| VIKI-R | GRPO (RFT after SFT) | Multi | Outcome | Multi | Embodied Multi-Robot Cooperation (VIKI-Bench) | Rule + Model | No |
| STeCa | DPO (RFT) | Single | Both | Multi | Embodied/Household | Rule/MC | Environment Actions |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework | Domain |
|---|---|---|---|---|---|---|
| FaithMed |  |
2026.7 | CMU | Paper | veRL + verl-agent | Medical |
| Gene-Disease-Curation |  |
2026.2 | Academic | Paper | Custom | Medical |
| MedSAM-Agent |  |
2026.2 | CUHK/Tencent | Paper | Custom | Medical |
| ChemCraft |  |
2026.1 | Peking University / IDEA | Paper | veRL | Chemistry |
| Doctor-R1 |  |
2025.12 | Tsinghua (thu-unicorn) | Paper | veRL | Medical |
| Alpha-R1 |  |
2025.12 | SJTU / FinStep.AI / StepFun | Paper | Custom | Financial |
| OS-R1 |  |
2025.8 | ISCAS | Paper | Custom | OS/Systems |
| MMedAgent-RL |  |
2025.8 | Unknown | paper | Unknown | Medical |
| MedResearcher-R1 |  |
2025.8 | Ant Group (AQ-MedAI) | Paper | Custom | Medical |
| LegalDelta |  |
2025.8 | Northeastern University (NEUIR) | Paper | Custom | Legal |
| DoctorAgent-RL |  |
2025.5 | UCAS/CAS/USTC | Paper | RAGEN | Medical |
| Biomni |  |
2025.3 | Stanford University (SNAP) | Paper | Custom | Biomedical |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| FaithMed | SFT (LLaMA-Factory) + agentic RL w/ process reward | Single | Both | Multi | Faithful evidence-based medical QA (MedQA/MedMCQA/MedXpertQA/...) | Rule + Model (step-level faithfulness) | Yes (medcorp evidence search) |
| Gene-Disease-Curation | Process-supervised Multi-Agent RL | Multi | Both | Multi | Clinical gene-disease validity curation (ClinGen) | Model (process) + Rule (outcome) | Yes (agent-as-tool, evidence synthesis) |
| MedSAM-Agent | GRPO (via veRL) | Single | Both | Multi | Medical Image Segmentation | Model (clinical fidelity) | Yes (SAM/MedSAM2) |
| ChemCraft | SMILES-GRPO | Single | Both | Multi | Chemical LM orchestrating chemistry tools (molecular design/synthesis) | External (dense chemical) + Rule | Yes (chemical agent sandbox) |
| Doctor-R1 | Experiential Agentic RL | Multi | Both | Multi | Clinical inquiry & diagnosis | Model + Rule + safety veto | No |
| Alpha-R1 | GRPO | Single | Outcome | Multi | Alpha factor screening (with real-time news) | External (portfolio returns) + Model | Yes |
| OS-R1 | GRPO (via veRL) | Single | Outcome | Multi | Linux Kernel Tuning | Rule | Yes (LightRAG, kernel config) |
| MMedAgent-RL | Unknown | Multi | Unknown | Unknown | Unknown | Unknown | Unknown |
| MedResearcher-R1 | GRPO-based (SFT + Online RL) | Single | Outcome | Multi | Medical Deep Research (MedBrowseComp) | Rule + Model | Yes (Search/KG) |
| LegalDelta | GRPO (CoT-guided info-gain) | Single | Process | Multi | Legal Reasoning | Model + Rule | No |
| DoctorAgent-RL | GRPO | Multi | Both | Multi | Consultation/Diagnosis | Model/Rule | No |
| Biomni | TBD | Single | TBD | Single | scRNAseq/CRISPR/ADMET/Knowledge | TBD | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | Focus |
|---|---|---|---|---|---|
| AgentV-RL |  |
2026.4 | Academic | Paper | Agentic Verifier Reward Model |
| DataMind |  |
2026.4 | Zhejiang University (ZJUNLP) | Paper | Process Reward Model (DataPRM) |
| ARLArena |  |
2026.2 | UCLA | Paper | Stable Agentic RL (SAMPO) |
| Agent-RRM |  |
2026.1 | Academic | Paper | Reasoning Reward Model for Agents |
| ToolPRMBench |  |
2026.1 | Arizona State University | Paper | PRM Benchmark for Tool-Use |
| RLVR-World |  |
2025.5 | THU ML Group | Paper | RLVR for World Models |
| AgentProg |  |
2025.5 | MobileLLM | Paper | Progress Reward Model (ProgRM) |
| AgentPRM |  |
2025.2 | Cornell | Paper | Process Reward for Agents |
| Agentic-Reward-Modeling |  |
2025.2 | THU-KEG | Paper | Agentic Reward Agent |
| AgentRM |  |
2025.2 | THUNLP/Tsinghua | Paper | Generalizable Agent RM |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| AgentV-RL | RL (verl) training an agentic verifier | Single | Process | Multi | Tool-augmented deliberative verifier (reward model) | Model-Based | Yes (verifier invokes tools, e.g. code) |
| DataMind | RL w/ generative PRM (DataPRM) | Single | Process | Multi | Agentic data analysis (Python/SQL; ScienceAgentBench/DABench) | Model (PRM) + External (execution) | Yes (code-based multi-turn) |
| ARLArena | SAMPO (Stable Agentic Policy Optimization) | Single | Outcome | Multi | Stable agentic RL across web/embodied/math/game/search | External + Rule | Yes (code/web/search/embodied) |
| Agent-RRM | Agentic RL w/ trained reasoning RM | Single | Process | Multi | Reward model for agents (web nav, multi-hop QA) | Model-Based (RRM) | Yes (agentic trajectories) |
| ToolPRMBench | N/A (Benchmark) | Single | Process | Multi | Tool-Use | Rule/Model | Yes |
| RLVR-World | RLVR | Single | Outcome | Multi | World Modeling (Language/Video) | Model (verifiable) | No |
| AgentProg | Online RL w/ progress reward | Single | Process | Multi | GUI Agent Training | Model (ProgRM) | Yes |
| AgentPRM | PPO/DPO + PRM | Single | Process | Multi | ALFWorld/General | Model (PRM) | Yes |
| Agentic-Reward-Modeling | DPO/Best-of-N | Single | Outcome | Single | General Instruction | Model (Reward Agent) | Yes (Verification) |
| AgentRM | MCTS/RM-guided | Single | Outcome | Multi | 9 Agent Tasks | Model (regression PRM) | Yes |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| ToolSafe |  |
2026.1 | Academic (MurrayTom) | Paper | veRL |
| TROJail |  |
2025.12 | Academic (ACL 2026) | Paper | RAGEN + vLLM |
| SafeSearch |  |
2025.11 | Amazon Science | Paper | veRL |
| Jailbreak-R1 |  |
2025.6 | Academic (yuki-younai) | Paper | Custom |
| GuardReasoner-VL |  |
2025.5 | NUS (yueliu1999) | Paper | Custom |
| xJailbreak |  |
2025.1 | Academic | Paper | Custom |
| Auto-RT |  |
2025.1 | ICIP-CAS | Paper | Custom |
| RLbreaker |  |
2024.6 | Purdue | Paper | Custom |
| curiosity_redteam |  |
2024.2 | MIT | Paper | Custom |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| ToolSafe | Multi-task GRPO | Single | Process | Multi | Tool-Invocation Safety Guardrail | Rule + Model | Yes (tool monitoring) |
| TROJail | Multi-turn GRPO variant | Single | Both | Multi | Multi-turn Jailbreak Attack | Model (harmfulness judge) + Rule | Yes (target LLM) |
| SafeSearch | PPO (GAE/GRPO) | Single | Both | Multi | Safe QA/Search | Rule + Model | Search |
| Jailbreak-R1 | GRPO (3-stage: imitation→warm-up→progressive) | Single | Both | Multi | Red-teaming Prompt Generation | Model (judge) | Yes (target LLM) |
| GuardReasoner-VL | Online RL w/ rejection sampling | Single | Both | Multi | VLM Safety Guard (multimodal) | Rule + Model | No |
| xJailbreak | RL | Single | Outcome | Multi | Jailbreaking | Model (embedding) | Yes (iterative) |
| Auto-RT | PPO | Single | Outcome | Multi | Red Teaming | Model | Yes (strategy exploration) |
| RLbreaker | Custom PPO | Single | Outcome | Multi | Jailbreaking | Model | Yes (mutator selection) |
| curiosity_redteam | RL + Curiosity | Single | Outcome | Multi | Red Teaming | Model | Yes (iterative query) |
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| VTS |  |
2026.7 | UNC Chapel Hill / Sony | Paper | ms-swift |
| VSeek |  |
2026.7 | UT Austin (SwarmLab) | Paper | veRL |
| HyperEyes |  |
2026.5 | DeepExperience | Paper | Custom |
| ODE |  |
2026.5 | HKUST / CUHK / PKU | Paper | verl + rllm |
| ParaVT |  |
2026.5 | NTU / HKU / Tsinghua / MiroMind (LMMs-Lab) | Paper | AReaL |
| OpenSearch-VL |  |
2026.5 | CUHK / NTU / HKU / Multi-institution | Paper | rLLM/veRL/Megatron-LM |
| MTA-Agent |  |
2026.4 | Salesforce AI Research | Paper | Custom |
| Gen-Searcher |  |
2026.3 | Academic | Paper | rllm + verl |
| MM-DeepResearch |  |
2026.3 | Academic | Paper | veRL |
| PyVision-RL |  |
2026.2 | agents-x-project | Paper | veRL |
| Vision-DeepResearch |  |
2026.1 | Academic (ICML 2026) | Paper | rllm + verl |
| ARM-Thinker |  |
2025.12 | Shanghai AI Lab / InternLM | Paper | veRL |
| CodeDance |  |
2025.12 | ByteDance | Paper | veRL |
| DeepEyesV2 |  |
2025.11 | Xiaohongshu | Paper | Custom |
| Mini-o3 |  |
2025.9 | Mini-o3 team | Paper | veRL |
| VisionThink |  |
2025.7 | CUHK (dvlab-research) | Paper | veRL + EasyR1 |
| multimodal-search-r1 |  |
2025.6 | ByteDance/NTU | Paper | Custom |
| AutoVLA |  |
2025.6 | UCLA Mobility Lab | Paper | Custom |
| VDeepEyes |  |
2025.5 | Xiaohongshu/XJTU | Paper | veRL |
| CoSo |  |
2025.5 | NTU/Alibaba | Paper | Custom |
| Pixel-Reasoner |  |
2025.5 | University of Waterloo (TIGER-AI-Lab) | Paper | OpenRLHF |
| Visual-ARFT |  |
2025.5 | Shanghai AI Lab / SJTU | Paper | Custom |
| VTool-R1 |  |
2025.5 | UIUC | Paper | veRL + EasyR1 |
| OpenThinkIMG |  |
2025.5 | Academic (zhaochen0110) | Paper | OpenR1 |
| Chain-of-Focus |  |
2025.5 | Multi-institution | Paper | veRL |
| GRIT |  |
2025.5 | UC Santa Cruz (eric-ai-lab) | Paper | trl |
| AlphaDrive |  |
2025.3 | HUST/Horizon Robotics | Paper | Custom |
| VSC-RL |  |
2025.2 | Liverpool/Huawei/Tianjin/UCL | Paper | Custom |
| RL4VLM |  |
2024.5 | UC Berkeley | Paper | Custom |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| VTS | GRPO (multi-turn tree rollout w/ backtracking) | Single | Both | Multi | Grounded long-video QA (search video as an adaptive temporal tree) | Rule-Based (answer + format + evidence IoU) | Yes (zoom_in/zoom_out/shift/answer) |
| VSeek | GRPO | Single | Both | Multi | Long-video QA as multi-turn evidence retrieval (LongVideoBench/LVBench/Video-MME/MLVU) | Rule-Based (neuro-symbolic temporal-logic grounding) | Yes (ViCLIP+FAISS retrieval server) |
| HyperEyes | Dual-grained efficiency-aware RL | Single | Both | Multi | Parallel multimodal search (unified grounded search) | Custom + Rule | Yes (visual grounding + retrieval) |
| ODE | GRPO (async) + SFT cold-start | Single | Both | Multi | Visual-native multimodal deep search (9-tool harness) | External + Rule | Yes (web/image/visual search, code) |
| ParaVT | PARA-GRPO (Parseability-Anchored, Ratio-gAted) | Multi (main + parallel sub-agents w/ shared weights) | Both (outcome + targeted format) | Single-turn parallel | Long-video understanding (VideoMME/LongVideoBench/LVBench/MLVU/MMVU/Charades-STA) | Rule + Model | Yes (parallel video-window crop tools) |
| OpenSearch-VL | Multi-turn fatal-aware GRPO | Single | Outcome | Multi | Multimodal Deep Search (Qwen3-VL base) | Rule + Model (LLM judge) | Yes (text/image search, OCR, crop, sharpen, SR, perspective) |
| MTA-Agent | DAPO (w/ cached tool interactions) | Single | Outcome | Multi | Multimodal Deep Search (21K MTA-Vision-DeepSearch; 32B beats GPT-5 54.63%) | Rule/External | Yes (web search, web read, Google Lens, image search) |
| Gen-Searcher | GRPO (after SFT) | Single | Both | Multi | Search-augmented image-generation deep research | Model (dual text+image) | Yes (search, image search, browse, image-gen) |
| MM-DeepResearch | Multi-turn agentic GRPO | Single | Both | Multi | Multimodal agentic search baseline | Model (judge) + Rule | Yes (image/text search engines) |
| PyVision-RL | GRPO | Single | Both | Multi | Agentic image+video understanding w/ dynamic Python tooling | External + Custom | Yes (Python-as-tool, frame sampling) |
| Vision-DeepResearch | GRPO (after cold-start SFT) | Single | Both | Multi | Multimodal deep-research MLLM (dozens of turns) | External + Rule | Yes (visual+textual search, browse) |
| ARM-Thinker | GRPO (two-stage) | Single | Both | Multi | Agentic multimodal reward modeling (Think-Act-Verify) | External + Rule | Yes (zoom/crop, doc retrieval, validators) |
| CodeDance | GRPO/DAPO (agent-loop) | Single | Both | Multi | Executable visual reasoning (visual search/math/chart via code) | External + Custom | Yes (Python sandbox: crop/draw/plot) |
| DeepEyesV2 | Outcome RL | Single | Outcome | Multi | Multimodal Reasoning | Rule | Yes (Code exec, Web search) |
| Mini-o3 | GRPO | Single | Outcome | Multi | Visual Search (V*/HR-Bench) | Rule | Yes (image crop) |
| VisionThink | GRPO w/ LLM-as-Judge | Single | Outcome | Multi | Efficient VQA | Model (LLM-Judge) | Yes (hi-res request) |
| multimodal-search-r1 | GRPO | Single | Outcome | Multi | Multimodal Search | Rule | Yes (Search) |
| AutoVLA | GRPO (RFT after SFT) | Single | Outcome | Multi | Autonomous Driving (nuScenes/nuPlan/Waymo) | Rule (PDMS) | No |
| VDeepEyes | PPO/GRPO | Multi | Process | Multi | VQA | All | Yes |
| CoSo | Soft RL (counterfactual) | Single | Outcome | Multi | Android/Card/Embodied | Rule | Yes |
| Pixel-Reasoner | Curiosity-driven GRPO | Single | Both | Multi | Visual Reasoning (V*/TallyQA/Info-VQA) | Rule + Model | Yes (zoom/select-frame) |
| Visual-ARFT | GRPO (agentic RFT) | Single | Outcome | Multi | Multimodal Agentic Tool Use (MAT-Search/Coding) | Rule | Yes (Search/Python) |
| VTool-R1 | RFT (GRPO-based) | Single | Outcome | Multi | Chart/Table VQA | Rule | Yes (Python visual tools) |
| OpenThinkIMG | V-ToolRL (GRPO) | Single | Outcome | Multi | Chart Reasoning | Rule | Yes (GroundingDINO/SAM/OCR/crop) |
| Chain-of-Focus | AGAR (GRPO) | Single | Outcome | Multi | Visual Reasoning (V*) | Rule (outcome+format) | Yes (zoom-in) |
| GRIT | GRPO-GR (Grounded Reasoning) | Single | Outcome | Single | Visual Reasoning (bbox) | Rule | Yes (bbox) |
| AlphaDrive | GRPO | Single | Outcome | Multi | Autonomous Driving | Rule (4 planning rewards) | No |
| VSC-RL | Variational RL | Single | Outcome | Multi | Mobile Device Control | Rule | Yes |
| RL4VLM | PPO | Single | Outcome | Multi | GymCards/ALFWorld | Rule | Yes |
⚠️ Note: The definition of "Self-Evolution" in the context of RL for LLM agents is still evolving and not yet well-established. This category currently collects works whose paper titles explicitly contain "self-evolving" or "self-evolution", where the agent improves itself through RL-driven feedback loops.
| Github Repo | 🌟 Stars | Date | Org | Paper Link | RL Framework |
|---|---|---|---|---|---|
| SEED |  |
2026.7 | Academic (jinyangwu) | Paper | veRL |
| OPID |  |
2026.6 | Academic (jinyangwu) | Paper | veRL + verl-agent |
| UCOB |  |
2026.6 | Academic (TU2021) | Paper | veRL |
| SIRI |  |
2026.6 | Academic | Paper | Custom (GiGPO) |
| world-knowledge |  |
2026.4 | HKUST / Tencent | Paper | Custom |
| ARISE |  |
2026.3 | George Washington University | Paper | veRL |
| Tool-R0 |  |
2026.2 | UIUC / ETH Zurich | Paper | veRL |
| MemSkill |  |
2026.2 | NTU/UIUC/UIC/Tsinghua | Paper | Custom |
| MemRL |  |
2026.1 | SJTU/Xidian/NUS/USTC/MemTensor | Paper | Custom |
| AgentEvolver |  |
2025.11 | Alibaba/Tongyi Lab | Paper | Custom |
| EvolveR |  |
2025.10 | KnowledgeXLab / Shanghai AI Lab | Paper | veRL |
| SEAgent |  |
2025.8 | Shanghai AI Lab / CUHK | Paper | Custom |
| R-Zero |  |
2025.8 | Tencent AI Seattle Lab / WashU / UMD | Paper | EasyR1 |
| Absolute-Zero-Reasoner |  |
2025.5 | Tsinghua (LeapLabTHU) / BIGAI / PSU | Paper | veRL |
| RAGEN |  |
2025.1 | RAGEN-AI | Paper | veRL |
| WebRL |  |
2024.11 | Tsinghua/Zhipu AI | Paper | Custom |
📋 Click to view technical details
| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |
|---|---|---|---|---|---|---|---|
| SEED | GRPO + self-evolving on-policy distillation (hindsight-skill SFT → OPD during RL) | Single | Outcome | Multi | ALFWorld / WebShop / Search-QA / EZPoints / Sokoban | Rule + External | Yes (interactive env actions, search) |
| OPID | On-policy skill distillation (hierarchical hindsight skills → token-level dense supervision) | Single | Both | Multi | ALFWorld / WebShop / Search-QA | Rule + External | Yes (interactive env actions, search) |
| UCOB | Credit-aware on-policy bidirectional self-distillation (skill-conditioned vs skill-free branches) | Single | Both | Multi | ALFWorld / WebShop / Search-QA | Rule (return-based) | Yes (interactive env actions, search) |
| SIRI | GiGPO + self-skill mining/distillation | Single | Both | Multi | Self-internalizing intrinsic skills (ALFWorld/WebShop) | External + Custom | Yes (interactive actions) |
| world-knowledge | Outcome-based RL (reward-free self-evolution) | Single | Outcome | Multi | Web Agents (WebVoyager/WebWalker; +20% on Qwen3-30B & Seed-OSS-36B) | Model (intrinsic; world-knowledge gain) | Yes (web pipeline for env-specific knowledge construction) |
| ARISE | Hierarchical RL (options + intra-option) w/ skill evolution | Single | Both | Multi | Reasoning w/ intrinsic skill library (7 Olympiad benchmarks) | External + Custom (skill-quality) | No (skill reuse over multi-step reasoning) |
| Tool-R0 | Self-play RL (generator+solver co-evolution) | Multi | Both | Multi | Self-evolving tool-learning from zero data | External + Custom | Yes (real tool/function calls) |
| MemSkill | PPO | Single | Process | Multi | QA/ALFWorld | Model (learned skills) | Yes |
| MemRL | RL-based (Q-value) | Single | Process | Multi | HLE/BigCodeBench/ALFWorld | Model (retrieval) | Yes |
| AgentEvolver | ADCA-GRPO | Single | Outcome | Multi | Social Game/Tool-use | Rule | Yes |
| EvolveR | GRPO (closed-loop online+offline) | Single | Outcome | Multi | Multi-hop QA (NQ/HotpotQA) | Rule | Yes (experience retrieval) |
| SEAgent | GRPO | Single | Outcome | Multi | Computer Use (OSWorld) | Model | Yes (Screenshot-based) |
| R-Zero | GRPO (Challenger + Solver co-evolution) | Multi | Outcome | Multi | Math/SuperGPQA/MMLU-Pro/BBEH | Rule (majority voting) | No |
| Absolute-Zero-Reasoner | TRR++ (Task-Relative REINFORCE++) | Single | Outcome | Single | Code/Math Reasoning (HumanEval/MBPP/LiveCodeBench) | Rule + learnability | Yes (Python exec) |
| RAGEN | PPO/GRPO (StarPO) | Single | Both | Multi | TextGame | All | Yes |
| WebRL | Actor-Critic RL + ORM | Single | Outcome | Multi | Web Navigation (WebArena) | Model (ORM) | Yes (Web browsing) |
| Github Repo | 🌟 Stars | Date | Org | Task |
|---|---|---|---|---|
| SETA |  |
2026.7 | CAMEL-AI.org | Terminal agents (4,500+ verifiable Docker envs, Terminal-Bench format; Paper) |
| OpenAgent |  |
2026.7 | Nanjing University (LAMDA-NeSy) | Tool-use generalization sandbox (query/schema/observation/domain shifts; Paper) |
| MobileGym |  |
2026.5 | Academic | Mobile GUI (Android sim, verifiable, parallel) |
| AEnvironment |  |
2026.5 | Ant Group (inclusionAI) | Agentic RL Env Platform (MCP, AReaL-integrated, TAU2/SWE/Terminal-Bench) |
| Gym-Anything |  |
2026.4 | CMU L3 Lab | Computer Use (200+ apps) |
| OpenSandbox |  |
2026.3 | Alibaba | Code/GUI/Agent Eval |
| OpenEnv |  |
2026.3 | Meta (PyTorch) | Chess/Arcade/Finance |
| open-trajectory-gym |  |
2026.3 | Individual | CTF/Security |
| Agent-World-Model |  |
2026.2 | Snowflake AI Research | Tool-use (1,000 MCP synthetic envs) |
| TermiGen |  |
2026.2 | UCSB | Terminal/Tool-use (3,500+ envs) |
| VisGym |  |
2026.1 | UC Berkeley | Multimodal/VLM (17 envs) |
| NeMo-Gym |  |
2026.1 | NVIDIA | Multi-step/Multi-turn |
| VISTA-Gym |  |
2025.11 | Texas A&M / Emory / KAUST | Tool-integrated visual reasoning (VLM) |
| LoCoBench-Agent |  |
2025.11 | Salesforce AI Research | SWE |
| Simia-Agent-Training |  |
2025.10 | Microsoft | ToolUse/API |
| PaperArena |  |
2025.9 | University of Science and Technology of China | ScientificLiteratureQA |
| enterprise-deep-research |  |
2025.9 | Salesforce AI Research | DeepResearch |
| meta-agents-research-environments |  |
2025.9 | Meta (FAIR) | Gaia2 / Multi-universe |
| BrowseComp-Plus |  |
2025.8 | University of Waterloo | Deep Research Eval |
| MCP-Bench |  |
2025.8 | Accenture | MCP Tool-use (28 servers) |
| MCPVerse |  |
2025.8 | Individual | MCP Tools (550+) |
| CompassVerifier |  |
2025.7 | Shanghai AI Lab | Reasoning |
| tau2-bench |  |
2025.6 | Sierra Research | Tool-Agent-User |
| Mind2Web-2 |  |
2025.6 | Ohio State University | Web |
| MCP-Universe |  |
2025.5 | Salesforce AI Research | MCP Tool-use |
| gem |  |
2025.5 | Sea AI Lab | Math/Code/Game/QA |
| MLE-Dojo |  |
2025.5 | GIT, Stanford | MLE |
| R2E-Gym |  |
2025.4 | UC Berkeley/ANU | SWE |
| SWE-smith |  |
2025.4 | Princeton/Stanford/SWE-bench | SWE |
| atropos |  |
2025.4 | Nous Research | Game/Code/Tool |
| InternBootcamp |  |
2025.4 | InternBootcamp | Coding/QA/Game |
| loong |  |
2025.3 | CAMEL-AI.org | RLVR |
| DataSciBench |  |
2025.2 | Tsinghua | data analysis |
| reasoning-gym |  |
2025.1 | open-thought | Math/Game |
| llmgym |  |
2025.1 | tensorzero | TextGame/Tool |
| SWE-Gym |  |
2024.12 | UC Berkeley/UIUC/CMU/Apple | SWE |
| debug-gym |  |
2024.11 | Microsoft Research | Debugging/Game/Code |
| gym-llm |  |
2024.8 | Rodrigo Sánchez Molina | Control/Game |
| AgentGym |  |
2024.6 | Fudan | Web/Game |
| tau-bench |  |
2024.6 | Sierra | Tool |
| appworld |  |
2024.6 | Stony Brook University | Phone Use |
| android_world |  |
2024.5 | Google Research | Phone Use |
| TheAgentCompany |  |
2024.3 | CMU, Duke | Coding |
| LlamaGym |  |
2024.3 | Rohan Pandey | Game |
| visualwebarena |  |
2024.1 | CMU | Web |
| LMRL-Gym |  |
2023.12 | UC Berkeley | Game |
| OSWorld |  |
2023.10 | HKU, CMU, Salesforce, Waterloo | Computer Use |
| webarena |  |
2023.7 | CMU | Web |
| AgentBench |  |
2023.7 | Tsinghua University | Game/Web/QA/Tool |
| WebShop |  |
2022.7 | Princeton-NLP | Web |
| ScienceWorld |  |
2022.3 | AllenAI | TextGame/ScienceQA |
| factorio-learning-environment |  |
2021.6 | JackHopkins | Game |
| alfworld |  |
2020.10 | Microsoft, CMU, UW | Embodied |
| jericho |  |
2018.10 | Microsoft, GIT | TextGame |
| TextWorld |  |
2018.6 | Microsoft Research | TextGame |
- CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution (ACL 2026, AMAP/Alibaba — repo created but code not yet released)
- JoyAgents-R1: Joint Evolution Dynamics for Versatile Multi-LLM Agents with Reinforcement Learning
- Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning
- Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning
- Acting Less is Reasoning More! Teaching Model to Act Efficiently
- Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
- ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
- Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward
- MUA-RL: MULTI-TURN USER-INTERACTING AGENTREINFORCEMENT LEARNING FOR AGENTIC TOOL USE
- Understanding Tool-Integrated Reasoning
- Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
- Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
- SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents
- WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents
- EnvX: Agentize Everything with Agentic AI
- UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning
- UI-Venus Technical Report: Building High-performance UI Agents with RFT
- Agent2 : An Agent-Generates-Agent Framework for Reinforcement Learning Automation
- Adversarial Reinforcement Learning for Large Language Model Agent Safety
- Learning to Refine: An Agentic RL Approach for Iterative SPARQL Query Construction
- InfoFlow: Reinforcing Search Agent Via Reward Density Optimization
Made with ❤️ by the AgentsMeetRL community
