Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Relax (Reinforcement Engine Leveraging Agentic X-modality) is a high-performance reinforcement learning post-training framework open-sourced by the Xiaohongshu AI Infra Team for multimodal large language models. Built on Ray Serve with a service-oriented architecture, it uses Megatron-LM as the training backend and SGLang as the inference engine, and through its TransferQueue data transfer system it decouples training and inference so that Rollout, Actor, ActorFwd, Reference, and Advantages can each run on independent GPU clusters. The framework supports three execution modes -- Colocate for memory-constrained strict on-policy training, Fully Async for maximum throughput with configurable staleness, and a Hybrid mode that streams data via TransferQueue while running reference/actor_fwd/advantages in-process on the actor. It ships PPO, GRPO, GSPO, SAPO, CISPO, and On-Policy Distillation out of the box, with pluggable rewards and a built-in GenRM (LLM-as-judge) mode, and is designed for end-to-end omni-modal post-training across text, vision, and audio, including Qwen3, Qwen3-VL, Qwen3.5, Qwen3-Omni, Qwen3.6, GLM5, Kimi K2.6, and dots.mocr model families. Elastic Rollout Scaling lets operators grow or shrink inference engines mid-training via HTTP REST API, and production tooling includes a HealthManager for auto-recovery, a centralized Metrics Service (WandB/TensorBoard/ClearML), and Apprise-based real-time notifications. The recommended install path is an official Docker image bundling CUDA, PyTorch, Megatron-LM, SGLang, and Ray.