HRM-Text is a 1B-parameter text generation model and full pretraining framework from Sapient Intelligence, built on the Hierarchical Reasoning Model (HRM) architecture. Its central claim is efficiency: the project positions foundation-model pretraining as accessible for roughly $1,000, reporting 130-600x less compute and 150-900x less data than conventional scaling-based training. Rather than shipping only weights, HRM-Text provides the end-to-end recipe — data pipeline, training code, evaluation, and checkpoint tooling — so researchers can pretrain a capable model from scratch. ## Hierarchical Recurrent Architecture HRM-Text is built on a hierarchical recurrent architecture strengthened by task completion and latent-space reasoning, a departure from the standard decoder-only transformer scaling recipe. The design emphasizes reasoning in latent space, and the framework integrates PrefixLM sequence packing, FlashAttention 3 kernels, and PyTorch FSDP2 for distributed training. Because the attention path depends on FlashAttention 3, Hopper-class GPUs (H100) are the expected training target. ## Two Reference Sizes The project publishes two reference configurations with reproducible cost and time estimates. The L size (0.6B parameters) trains on 8 H100s in a single node in about 50 hours (~$800), while the XL size (1B parameters) uses 16 H100s across two nodes in about 46 hours (~$1,472). These concrete recipes make the efficiency claims verifiable rather than aspirational, and they lower the barrier for academic labs and independent researchers who lack hyperscaler budgets. ## Benchmark Results The reference runs report strong results for models of this scale. The XL (1B) model reaches 84.7% on GSM8k, 56.5% on MATH, 82.3% on DROP, 60.7% on MMLU, 81.9% on ARC-C, 63.4% on HellaSwag, 72.4% on Winogrande, and 86.2% on BoolQ. The L (0.6B) model trails but remains competitive, for example 77.6% on GSM8k and 56.6% on MMLU. The emphasis on math and reasoning benchmarks aligns with the architecture's latent-reasoning focus. ## Complete Pretraining Toolchain HRM-Text is designed to be run, not just read. It pairs with a companion data_io pipeline for cleaning, tokenizing, and stratified-sampling the pretraining corpus, and it supports single-node and multi-node layouts with shared storage. A published Docker image bundles the full environment including PyTorch, CUDA, and FlashAttention 3, and training integrates Weights & Biases for metric tracking. The workflow covers NCCL communication checks, torchrun launch commands, and checkpoint conversion. ## Assessment HRM-Text is a notable entry in efficient pretraining, offering a reproducible path to a reasoning-capable 1B model at a fraction of typical cost. Its main constraints are practical: it targets Hopper-class hardware due to FlashAttention 3, and reproducing the headline results still requires access to 8-16 H100s. The strong self-reported reasoning benchmarks warrant independent verification, but the open framework, transparent cost estimates, and complete tooling make it a valuable resource for researchers exploring alternatives to brute-force scaling.