Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Xing4.0-29B-A4B is a mixture-of-experts language model released by China Telecom Artificial Intelligence Technology Co., Ltd. as the latest Xing-series model, the successor to TeleChat. It has 29B total parameters with only 4B activated per token, and natively supports a 256K-token context window extensible to 512K. It is built on what the team calls an "Agent-Oriented Architecture" of mHC, Multi-head Latent Attention (MLA), and multi-token prediction (MTP), aimed at multi-step planning and tool calling. It has 40 layers, hidden size 3584, dense FFN intermediate size 9216, expert intermediate size 1024, 64 routed experts plus 1 shared expert, and 4 active experts per token. The team reports it is the first model of this scale trained entirely on Ascend NPU with MindSpore, on Ascend 910C clusters, and reports about a 96% training-throughput gain from combined MoE communication optimization, selective recomputation, graph-operator fusion, and custom Ascend C operators. Weights are ungated and Apache-2.0 licensed, published as 41 safetensors shards on Hugging Face, alongside separate FP8 and GGUF (IQ4_NL-quantized) variants, both also ungated and Apache-2.0. Fine-tuning is supported through LLaMA-Factory and MindFormers, the latter adapted for Ascend Atlas 800T A3 clusters, with multi-chip deployment via BAAI FlagOS. Inference support for vLLM, SGLang, TensorRT-LLM, llama.cpp, and KTransformers has been submitted as pull requests that are not yet merged upstream, so users install from the PR branches until they land. The README describes "targeted adaptation and format alignment," not official integration, with agent frameworks OpenCode, Claude Code, OpenClaw, and Hermes. The README's three-way table against Gemma4-26B-A4B and Qwen3.6-35B-A3B shows Xing4.0-29B-A4B leading both rivals on Claw-Eval, Terminal-Bench 2.1, and DeepresearchBII, and nearly matching Qwen3.6-35B-A3B on SWE-bench Verified (75.00 vs 76.00) while well ahead of Gemma4-26B-A4B there; it trails Qwen3.6-35B-A3B on AIME2026, SWE-bench Multilingual, and Tau3-Bench, and trails on IFBench and AA.LCR, where Gemma4-26B-A4B scores highest. The team's acknowledgments credit DeepSeek's architecture design with improving training stability and efficiency.