Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
This repo contains the source code for the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning"
| Date | Stars |
|---|---|
| 2026-07-31 | 377 |
| 2026-08-01 | 377 |
| 2026-08-02 | 377 |
| 2026-08-06 | 377 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# ES-at-Scale: Evolution Strategies at Scale <div align="center"> <img src="assets/es_at_scale_final.gif" alt="ES-at-Scale" width="100%"/> </div> ## Overview **ES-at-Scale** is an open-source framework for fine-tuning large language models using **Evolution Strategies (ES)** — a fully backpropagation-free, massively parallelizable alternative to RL-based training methods like PPO and GRPO. The implementation is based on the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning" (https://arxiv.org/abs/2509.24372). ES-at-Scale performs **direct optimization in the full parameter space**: - **No backpropagation** - **No optimizer states** - **No activations stored** - **No dimensionality reduction or low-rank adapters** Training is built on **Ray** for distributed execution and **vLLM** for high-throughput inference, enabling efficient multi-GPU rollout evaluation at scale. Feel free to join the ES fine-tuning forum in [Discussions](https://github.com/VsonicV/es-fine-tuning-paper/discussions). For the older version of the codes that were used to generate the original experimental results in the paper, please see `/archive` (with corresponding documentations inside). ### News 06/18/2026: :fire::fire::fire: A new open-source library **ES-at-Scale** with an extensible interface and a more stable instrastructure is released! :rocket::rocket::rocket: 10/27/2025: :fire::fire::fire: An accelerated version with **10X+ speed-up** in running time is added to the repo! :rocket::rocket::rocket: --- <!-- ## 🎉 What's New --- --> ## At a glance **ES-at-Scale** is free to use, modify, and build on. - **Massively parallel**: each perturbation (population member) can be evaluated independently - **Works with non-differentiable rewards**: symbolic graders, program execution, discrete checks - **Inference-only systems**: leverage vLLM throughput instead of training-time backprop --- # Training Models of Any Size That Fit Your Hardware A major design goal of this repository is **model‑size flexibility** — and, in turn, **democratizing LLM fine‑tuning**. Using this exact implementation we have successfully trained **0.5B, 3B, 7B, 14B, 32B, and 72B parameter models** with no architectural changes. > If the model **fits in your GPUs under vLLM**, it can be fine‑tuned with this ES framework. Because there are no gradients, optimizer states, or stored activations, the scaling challenge shifts from gradient computation to **inference throughput**. Smaller teams can fine‑tune large models on inference‑optimized clusters, with arbitrary (including non‑differentiable) reward functions. --- ## Requirements - Python **3.12** is reccommended. Other python versions are untested and may not work - CUDA-enabled GPUs (**multi-GPU strongly recommended**) - Linux environment - CUDA, PyTorch, Ray, and vLLM properly installed --- ## Setup ### 1. Create a virtual environment > Python 3.12 is supported and tested. Other versions are untested and may not work. ```bash python3.12 -m venv es source es/bin/activate ``` ### 2. Install the repository From the repository root: ```bash pip install -e . ``` ### 3. Install additional dependencies ```bash pip install wandb ``` ### 4. Install math evaluation dependencies ```bash pip install math-verify pip install pylatexenc pip install latex2sympy2_extended ``` > **Note** > Ensure `nvidia-smi` works correctly and that Ray detects all available GPUs. --- ## Usage ### Example: Fine-Tuning with `train.py` `train.py` is a **task-specific example** showing how to use `EvolutionStrategiesTrainer`. It bundles two ready-to-run tasks — `countdown` (the default) and `math` — selected via `--task`. Each task wires in its own reward function, prompt template, and on-disk HuggingFace datasets. To use ES on your own task, copy `train.py` and replace those three components — the trainer itself is fully task-agnostic. `--task` selects the bundled configuration: - `countdown
Excerpt of 24,214 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:949bb5c21834face, llm:Description: 'source code for the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning"' — fine-tuning LLMs using evolution strategies.
matched fp:949bb5c21834face, llm:Description: 'source code for the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning"' — fine-tuning LLMs using evolution strategies.
matched fp:949bb5c21834face, llm:Description: 'source code for the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning"' — fine-tuning LLMs using evolution strategies.