Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
ZGCM-1 is a 7.39B-parameter dense Transformer trained from scratch by Zhongguancun Academy and the Zhongguancun Institute of Artificial Intelligence, built for mathematical reasoning and tool-assisted search, supporting both internal thinking and direct-response modes in one model with a 262,144-token (256K) context window. Its hybrid attention combines 27 gated sliding-window layers with 5 global layers across 32 total layers. Training follows three stages detailed in the accompanying technical report (arXiv 2609.13356): pretraining on roughly 4.19T tokens with hybrid FP8 precision and the Muon optimizer; mid-training on roughly 600B tokens extending context from 16K to 64K to 256K; and supervised fine-tuning that jointly trains general and agentic behavior on mixed thinking/direct-response examples with execution-verified trajectories. The report also explores mixed reinforcement learning for math and code, and the repository ships a separate rl/ directory alongside data-process/, pretrain/, midtrain/, and sft/. Evaluated on the 256K SFT checkpoint in thinking mode, ZGCM-1 scores 97.13% on MATH-500, 75.00% on AIME 2026, and 70.42% on HMMT 2025; the README credits it with the best average rank across the report's 14 reasoning benchmarks among seven compared 7B-8B models, though in the shown table it trails DeepSeek-R1-0528-Qwen3-8B on AIME 2024 and 2025 and MiniCPM4.1-8B on AIME 2024. On agentic search, it reaches 63.09% on WebWalkerQA, 19.43% on BrowseComp, and 62.00% on Binary Function Search, matching function entry points with Ghidra tools. The gated sliding-window/global attention design gives 3.94x training throughput at 256K versus full attention in the report's architecture experiments, with an estimated ~4.2x improvement in 16K pretraining time-to-loss from combined architecture, precision, optimizer, and normalization gains. Weights are on Hugging Face at zgcagi/ZGCM-1-7B (requires trust_remote_code) alongside a companion dataset at zgcagi/ZGCM-1-Data, which is access-gated; the training code and the model weights are both MIT-licensed, while the dataset is listed under its own terms. The README notes that researcher-directed AI agents contributed throughout development.