DeepSeek-V3 is a large Mixture-of-Experts (MoE) language model from DeepSeek-AI, released as open weights alongside a detailed technical report. It has drawn intense attention from the AI community — over 100,000 GitHub stars — largely because it reaches performance comparable to leading closed-source models while documenting, in unusual detail, how it was trained efficiently and at relatively low cost. The repository hosts the inference code, weights pointers, and the paper rather than a product, making it a reference point for anyone studying or deploying frontier open models. ## Architecture The model has 671 billion total parameters but activates only 37 billion per token, the defining property of a sparse MoE design: capacity scales without a proportional increase in per-token compute. DeepSeek-V3 builds on the architecture validated in DeepSeek-V2, using Multi-head Latent Attention (MLA) to compress the key-value cache for cheaper inference and DeepSeekMoE for the expert layers. Two notable innovations are an auxiliary-loss-free strategy for load balancing across experts — avoiding the performance degradation that balancing losses usually introduce — and a multi-token prediction (MTP) objective that strengthens the model and can also be reused for speculative decoding to speed up inference. ## Training Efficiency DeepSeek-V3's most discussed contribution is cost. The team pre-trained it on 14.8 trillion tokens using an FP8 mixed-precision framework, reporting that this was the first validation of FP8 training at this scale. Through co-design of algorithms, frameworks, and hardware, they nearly achieved full overlap of computation and cross-node communication, completing pre-training in roughly 2.7 million H800 GPU-hours. They also report a remarkably stable run with no irrecoverable loss spikes or rollbacks — a meaningful claim for a model of this size, since instability is a common and expensive failure mode at the frontier. ## Capabilities The model supports a 128K-token context window and performs strongly across reasoning, math, and code benchmarks, where the team reports results competitive with top proprietary systems. A distinctive post-training step distills long chain-of-thought reasoning patterns from the DeepSeek-R1 series into the standard V3 model, improving its reasoning while keeping output style and length under control. ## Practical Use DeepSeek-V3 is accessible several ways: open weights on Hugging Face for self-hosting, a hosted chat interface, and an API platform. Running it locally is non-trivial — 671B parameters demand serious multi-GPU infrastructure — so many teams reach it through inference engines that provide optimized support, or via the official API. The code is MIT-licensed, while the model weights are governed by a separate model license that should be reviewed for commercial deployments. ## Considerations The trade-offs are mostly about scale. Self-hosting requires substantial GPU memory and serving expertise, putting full local deployment out of reach for individuals and small teams. As a large MoE model, getting good throughput depends on a capable inference stack and careful configuration. And while the technical report is unusually transparent, the released artifacts are weights and inference code rather than the full training pipeline. For researchers and organizations seeking a top-tier open-weight model — and a well-documented blueprint for efficient large-scale training — DeepSeek-V3 stands out as a landmark release.