Moonshot AI Releases Kimi K3 Open Weights
Moonshot AI released the open weights for Kimi K3 on July 27, making its 2.8 trillion-parameter multimodal mixture-of-experts model available under the Kimi K3 License. The Hugging Face model card lists 104 billion activated parameters, 896 experts with 16 selected per token, native vision support, and a 1,048,576-token context length. The original ingestion record reports a 1.56 TB download.
Moonshot AI released the open weights for Kimi K3 on July 27, publishing a 2.8 trillion-parameter multimodal mixture-of-experts model on Hugging Face under the Kimi K3 License. The original ingestion record reports that the weight download is 1.56 TB, a size that makes local experimentation and deployment a substantial infrastructure undertaking.
According to Moonshot AI's Hugging Face model card, Kimi K3 has 104 billion activated parameters from a pool of 896 experts, selecting 16 experts per token. The model has 93 layers, a 160K-token vocabulary, native text and image support, and a maximum context length of 1,048,576 tokens. Its listed vision encoder is MoonViT-V2, with 401 million parameters.
Architecture and deployment details
Moonshot AI describes Kimi K3 as using Kimi Delta Attention (KDA), Attention Residuals (AttnRes), and a Stable LatentMoE design. The company states that the architecture combines 69 KDA attention layers with 24 Gated MLA layers, and uses MXFP4 weights and MXFP8 activations through quantization-aware training.
In its technical blog, Moonshot AI claims that KDA, AttnRes, a more sparse MoE configuration, and revised training and data recipes provide an approximately 2.5x improvement in scaling efficiency relative to Kimi K2. That is a company-reported performance and efficiency claim rather than an independently validated result.
The Decoder reports that the release also includes a technical report and open-sourced infrastructure components, including attention kernels, an MoE communication library, and tooling for operating AI agents at scale. The available source material does not provide independent benchmark methodology or reproducible third-party measurements for the model's broader capability claims.
What the weight release changes
Moonshot AI's model card frames Kimi K3 for long-horizon coding, tool use, knowledge work, and multimodal tasks. Those are intended-use claims from the publisher. The more immediately verifiable facts are the model's large sparse-MoE topology, long-context configuration, and quantized weight format.
Open-weight releases at this scale can broaden the scope for inference-engine optimization, expert-parallel serving research, quantization experiments, and evaluation on private workloads. In comparable deployments, however, total parameter count, storage footprint, communication overhead, and hardware support for low-precision formats often determine whether an open model is practical to run rather than merely available to download.
Kimi K3's release is notable because it makes an unusually large model available for research and deployment under its stated license. Teams evaluating it will need to review the license terms, validate claimed capability on their own tasks, and assess serving requirements against their available accelerator, storage, and interconnect capacity.
Key Points
- 1Moonshot AI released Kimi K3 weights, exposing a 2.8 trillion-parameter sparse MoE model for research, deployment, and independent evaluation.
- 2The model card lists 104 billion activated parameters and 16 selected experts, making expert routing and distributed inference central engineering considerations.
- 3Comparable open-weight deployments show that storage, quantization support, and interconnect bandwidth frequently constrain practical use of very large MoE models.
Scoring Rationale
Kimi K3 is a very large open-weight multimodal MoE release with a 1 million-token context limit and disclosed architectural details. It is highly relevant to teams researching large-scale inference, expert parallelism, long-context evaluation, and self-hosted frontier-scale models, although practical deployment is limited by its reported 1.56 TB weight footprint.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
