Snowflake Research
Explore Snowflake's initiatives and contribution to making enterprise AI easy, efficient, and trusted through open research and innovation.
Arctic RL
Arctic RL Overview
Arctic RL is a high-throughput reinforcement learning training and inference backend designed to integrate with existing RL frameworks, providing unified GPU backends and system optimizations for accelerated post-training of large language models.
Configuration Reference
Configure the Arctic RL client with backend, communication protocol, model, and GPU resource settings through the ArcticRLClientConfig model. This page documents all available configuration options for initializing and tuning the platform.
Client API
The ArcticRL Client API provides a unified interface for reinforcement learning training, sampling, and inference operations. It supports both HTTP and Ray communication protocols with automatic job initialization and lifecycle management.
Weight Synchronization
The `WeightSyncCoordinator` manages efficient transfer of model weights from training GPUs to inference replicas using NCCL topology, enabling thread-safe coordination between training and inference clients with lazy sender setup and HTTP-orchestrated synchronization.
Framework Integrations
Arctic RL integrates into existing RL frameworks rather than replacing them, with completed integrations into SkyRL and an in-progress integration into Verl, plus additional framework integrations planned for TRL, Axolotl, unsloth, and PrimeRL.
Training Pipeline & Processors
The Arctic RL training pipeline provides a composable, two-phase architecture for forward passes: post-forward processors compute derived outputs (like logprobs from logits), and loss functions compute scalar losses and metrics. Register processors by name to make them available at runtime via batch configuration.
DeepSpeed Worker
A Ray-managed single-GPU actor that handles DeepSpeed training, inference, and weight synchronization for reinforcement learning workloads. Supports distributed training with gradient checkpointing, ZeRO optimization, and optional Liger kernel acceleration.
Ray Server & Deployment
The Arctic RL platform uses Ray to manage distributed DeepSpeed workers and ArcticInference ReplicaPools for training, sampling, and log probability computation. This page covers the server architecture, initialization, and key deployment patterns including colocated GPU resource sharing.
Overview
Introduction to Arctic Platform
Arctic Platform is a modular framework that simplifies and accelerates post-training for large language models through unified GPU backends, optimized compute engines, and integrated pipelines. It offers reinforcement learning components, prompt deduplication, and efficient inference optimizations designed to integrate seamlessly with existing training frameworks.
Project Scope & Roadmap
Arctic Platform aims to cover the full post-training stack for LLMs behind a composable API, currently offering reinforcement learning components, prompt-deduplication optimization, and efficient inference with additional trainers and data pipelines coming next.
Contributing
Citation & License
Learn how to cite Arctic Platform in your research and understand the Apache 2.0 license governing the project.
AI Disclaimer & Acknowledgments
Arctic Platform discloses how AI tools were used in development and acknowledges the contributions of human and machine intelligence in building the codebase. This page explains our commitment to transparency regarding AI assistance in the project.
Getting Started
Quickstart
Get started with Arctic Platform by installing the package and following the provided recipes for RL training with existing frameworks like Verl.
Installation
Arctic Platform can be installed from PyPI for the latest stable release or from source for the latest development version. Installation in editable mode is recommended for active development.
ZoRRo Train
ZoRRo Train Overview
ZoRRo Train eliminates redundant prompt computation in RL training by automatically deduplicating identical prompts at the attention layer, reducing 80–95% of wasted token computations while maintaining mathematical correctness. This transparent optimization delivers 2–4x speedup for high-deduplication scenarios through prompt grouping, optimized attention, and automatic reconstruction.
ZoRRo Train API Reference
Complete API reference for ZoRRo Train's core classes and methods for prompt deduplication in RL training. Covers DeduplicatedActor, ZoRRoTrain, and supporting utilities.
Performance Benchmarks
ZoRRo Train achieves significant speedups through prompt deduplication, with expected 2–4x improvements under high deduplication scenarios and dependencies on hardware, sequence length, and attention implementation.
How Prompt Deduplication Works
Prompt deduplication removes redundant computation in RL training by identifying sequences with identical prompts and processing them together, eliminating up to 82% of wasted token processing while maintaining mathematical correctness for gradients.
Supported Models
ZoRRo Train currently supports Qwen3 model families including dense, MoE, and hybrid architectures, with plans to expand support to additional models.
Testing & Correctness Verification
ZoRRo Train includes comprehensive CPU and GPU tests to verify deduplication correctness, gradient accuracy, and performance. Run tests with pytest and benchmarks with dedicated scripts.
Getting Started with ZoRRo Train
ZoRRo Train is a prompt deduplication optimization that eliminates redundant computation in RL training by identifying identical prompts and processing them once. Learn how to install, configure, and use the system to achieve 2-4x speedups in typical RLHF scenarios.
Recipes
Long-Context QA with Arctic RL
Train Qwen3-32B on long-context multi-hop QA tasks using GRPO with Arctic RL and the ZoRRo trainer. This recipe covers environment setup, data preparation, and distributed training across multiple nodes.
Simple Single-GPU GRPO (GSM8K)
A minimal end-to-end Arctic RL recipe for GRPO training with Qwen3-1.7B on GSM8K using a single GPU, served by Arctic RL with the ZoRRo trainer and no Ray cluster setup required.
Arctic RL Recipes Overview
Arctic RL Recipes provides end-to-end training recipes for reinforcement learning models using Arctic RL, with examples ranging from simple single-GPU setups to specialized tasks like text-to-SQL and long-context question answering.
Text-to-SQL (BIRD Benchmark)
GRPO training for Qwen3-32B on the BIRD SQL benchmark using Arctic RL and the ZoRRo trainer. This recipe demonstrates large-scale reinforcement learning for text-to-SQL generation with execution-match rewards on a 4-node, 32-GPU setup.