**AxisRL** is a post-training framework for the case where reinforcement learning stops being a single-turn problem. It sits between SGLang for rollout and Megatron for training, and takes responsibility for everything in between: multi-turn agent trajectories, weight synchronization, data movement, resource scheduling, and — unusually — reproducible debugging of the places where those two engines silently disagree. The repository was published by **XYZ AI Lab** on **23 July 2026** and has reached roughly **1,025 stars** with commits through 3 August. It is Apache-2.0 licensed and requires Python 3.12+. ## Why Agentic RL Breaks Existing Frameworks In single-turn RLHF, a prompt goes in, a completion comes out, a reward is computed, and the trainer consumes it. In agentic RL, a model interacts with a long-lived environment, calls tools, observes results, updates its context, and receives a reward only after several turns — the project cites workflows with **300+ turn trajectories** at hundreds-of-billions-parameter scale. That changes what the framework is responsible for. It must coordinate rollout, environment state, tool calls, verifiers, reward collection, sample construction and weight sync. And it must make all of that observable, because small differences in tokenization, chat templates, logprobs, expert routing, sequence packing or weight versions do not fail loudly — they surface later as loss spikes, reward instability and rollout-trainer mismatch. AxisRL's stated design goals map directly onto that: | Goal | Problem | Approach | |---|---|---| | Flexibility | Agent workflows differ in control flow, tools, rewards and resource needs | Recipes for task logic; white-box and black-box integration; resource groups for heterogeneous components | | Efficiency | Long-tail trajectories and tool latency leave rollout or training idle | Partial rollout, thin control plane, handle-based data movement, prefix-tree merge, MagiAttention | | Observability | Rollout and trainer paths diverge silently | Boundary tests, mismatch analysis, routing replay checks, spike replay | ## White-Box and Black-Box The integration split is the framework's most practical idea. **White-box** recipes express the environment loop inside AxisRL — appropriate for math, search and simple tool environments, where the user supplies the environment, tools, verifier and reward and the framework handles the agent loop and sample construction. **Black-box** recipes invert it. An existing harness — OpenHands, a browser agent, anything already built — calls the model through an **OpenAI-compatible proxy**, and AxisRL captures the model inputs, outputs, metadata and rewards flowing through that proxy to build trainable samples. This means an agent harness can be trained without being rewritten into the framework's abstractions, which is where most RL frameworks demand the integration work. Four recipes ship: GSM8K with GRPO, GSM8K with PPO, Search-R1 for retrieval-augmented multi-turn search, and the black-box OpenHands/E2B path. Configurable objectives cover PPO, GRPO/GRPO2, GSPO, TOPR and TIS. ## Consistency as a First-Class Feature The part of AxisRL that reads least like a research artifact is its debugging surface. Three mechanisms stand out. **Rollout Routing Replay (R3)** addresses MoE post-training specifically: the router that fired during rollout and the router that fires during training can pick different experts for the same token, and that mismatch shows up as unstable KL and loss. R3 replays the rollout's routing decisions on the trainer side, with the routing payloads moving through the handle-based data path so the driver never becomes a heavy-data relay. **Mismatch analysis** compares token-level differences across rollout and trainer paths, backends and configurations — designed to distinguish broad drift from a handful of outliers from something concentrated in one token range or context layout. **Spike replay** snapshots weights, optimizer state, data and routing information *before* a suspicious update, so a loss spike can be replayed and inspected rather than waited for again. Anyone who has tried to reproduce a non-deterministic training failure will recognize what this is worth. Alongside those, **partial rollout** hands completed or partially completed samples to the trainer early, so a few slow trajectories with many tool calls don't hold the whole batch hostage. ## Caveats **It is two weeks old with 2 contributors and 22 forks.** The star count outpaces the community by a wide margin. There is no track record of external users hitting problems and getting them resolved. **The black-box recipe — arguably the headline feature — is marked work in progress.** The README states the config, launch scripts and proxy interfaces may change. Building on that path today means accepting churn. **The hardware floor is high.** The recommended environment is a Docker image bundling SGLang, Megatron Core, MagiAttention, Ray and CUDA dependencies, and the default recipes assume a GPU machine sized for their parallelism. The optional data download includes multi-billion-parameter models. This is not a framework that starts on a laptop. **No public benchmarks or case studies yet.** Claims about 300+ turn trajectories and hundreds-of-billions-parameter training come from the project's description of its own internal use; "more real-world agent recipes and public case studies" sits on the roadmap rather than in the repository. **It inherits two heavy dependencies.** SGLang and Megatron remain the serving and training engines. AxisRL's value is the system layer around them, which also means its ceiling and its upgrade path are tied to both. ## Verdict The framing worth taking from AxisRL is that agentic post-training has become a **systems** problem more than an algorithms one. The objectives — PPO, GRPO, GSPO — are configurable options here, almost commodity. What gets a dedicated architecture is routing consistency, packing semantics, weight-sync correctness and the ability to replay a spike. That is a fair description of where the difficulty actually lives at this scale. Teams already running SGLang and Megatron for multi-turn agent RL are the clear audience, and the black-box proxy path is worth evaluating even in its unfinished state, because training an existing harness without rewriting it is a genuinely different offer. Everyone else should treat this as an early read on how the tooling around agentic RL is consolidating, and wait for the case studies.