Baidu Updates vLLM-Kunlun XPU Plugin
Baidu's vLLM-Kunlun repository lists version 0.25.1 as under development, adding support for Qwen3.5, Qwen3.5-MoE, Gemma4, GLM MoE DSA, and DFlash speculative decoding. The community-maintained plugin connects standard vLLM deployments to Kunlun XPU hardware through the vLLM hardware-plugin interface; the repository's latest merge commit is dated July 30, 2026.
Baidu's vLLM-Kunlun GitHub repository lists version 0.25.1 as under development, with additions for Qwen3.5, Qwen3.5-MoE, Gemma4 text and multimodal models, GLM MoE DSA, and DFlash speculative decoding. The repository's latest merge commit is dated July 30, 2026. It describes vLLM-Kunlun as a community-maintained hardware plugin for running vLLM on Kunlun XPU hardware.
The repository labels v0.25.1 as a development version, not a stable release. Its project news also records earlier 2026 work on fused mixture-of-experts inference for small batches, attention-metadata construction, Multi-LoRA inference, Multi-Token Prediction support for DeepSeek-V3.2, and several quantization formats.
Plugin architecture and model portability
A December 2025 Baidu developer article describes the project as an implementation of vLLM hardware-plugin standard RFC #11162. According to that article, the plugin separates the upstream vLLM codebase from the Kunlun XPU backend: developers install standard vLLM alongside the plugin rather than modifying vLLM core code for the accelerator.
The article describes a P800 execution path in which vLLM discovers and registers the plugin during engine initialization, then creates a Kunlun-optimized ModelRunner and loads plugin-specific model classes. Those classes call Kunlun-targeted kernels for low-level execution. The architecture is intended to confine adaptation work for new vLLM interfaces or model architectures to the plugin layer, according to Baidu's account.
This separation matters beyond a single accelerator. Hardware abstraction layers can reduce the maintenance burden created when upstream inference engines change APIs or add model architectures. In comparable inference stacks, the practical test is whether backend plugins remain current with upstream scheduler, attention, quantization, and model-executor changes without requiring forks of the core engine.
Kernels, debugging tools, and stated coverage
Baidu's developer article states that the project includes Kunlun-targeted fused operators, including Split_Norm_Rope and fused MoE implementations, made available through its xtorch_ops library. The same article reports that these operators were evaluated with DeepSeek, Qwen, Llama, and GLM models, and characterizes P800 throughput and latency as comparable with mainstream AI accelerators. That is a vendor-reported performance claim; the retrieved materials do not provide reproducible benchmark configurations, workloads, or comparison hardware.
The project also makes two accompanying tools available, according to Baidu:
- •torch_xray, for comparing layer-level outputs between GPU and P800 runs to locate numerical differences.
- •xpu_profiler, a profiler described as providing an operator timeline similar to Nvidia Nsight Systems.
The same developer article reports coverage of more than 20 model and multimodal-model families on P800, including Qwen, DeepSeek, Llama, GLM, InternVL, and GPT OSS.
For ML infrastructure teams evaluating non-CUDA inference targets, output validation and operator-level profiling are often as consequential as nominal model compatibility. A plugin can expose an upstream model architecture quickly, but deployment readiness also depends on numerical parity, memory behavior, batching performance, and observability under production request patterns.
The repository says vLLM-Kunlun was initially open sourced on December 8, 2025. The current development branch indicates that the project is continuing to track newer model families and inference features within the vLLM ecosystem.
Key Points
- 1The v0.25.1 development branch adds newer model families and speculative-decoding features to a vLLM backend for Kunlun XPU inference.
- 2Baidu describes RFC #11162 plugin separation as avoiding vLLM core modifications, reducing adaptation work when upstream interfaces change.
- 3For alternative accelerator deployments, reproducible performance data, numerical validation, and profiler output remain essential alongside claimed model compatibility.
Scoring Rationale
This is a notable inference-infrastructure update for teams deploying open models on Kunlun XPU hardware, particularly because it tracks current model and decoding features. Its practitioner impact is narrower than a core vLLM release because the plugin targets a specific accelerator ecosystem, and the available performance claims lack public benchmark detail.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
