D-Matrix Ships Corsair Inference Accelerator, Claims 10x Token Speed

d-Matrix said on June 9 that its Corsair inference accelerator has entered full production and begun shipping in volume to priority hyperscalers, neoclouds, and frontier AI labs, according to the company's announcement and a same-day PR Newswire release. Founder and CEO Sid Sheth said, "We built Corsair specifically for this moment, the Age of AI Inference." The company claims rack-level configurations pairing Corsair accelerators with GPUs generate tokens about 10x faster than GPU-only setups, at roughly 3x lower cost and up to 5x lower energy use for certain latency-sensitive inference workloads. Corsair is built on a TSMC 6-nanometer (N6) node in partnership with Alchip Technologies and is packaged as a server-pluggable, PCIe-based unit; d-Matrix's SquadRack reference design involves partners including Supermicro, Arista, and Broadcom.

The core technical claim rests on d-Matrix's Digital In-Memory Computing architecture, which pairs large SRAM pools directly with compute logic rather than relying on HBM/DRAM and PCIe transfers for the decode phase of inference. That design targets the industry's memory-wall bottleneck for real-time, low-latency LLM serving, at the cost of lower on-chip capacity for very large models unless mitigated by chiplet scaling or model partitioning. Gimlet Labs, an independent partner, published its own testing showing a 2-10x reduction in end-to-end request latency for speculative decoding when pairing Corsair with GPUs, reporting a drop from a 24-second baseline response to under two seconds in one configuration; this is the closest available third-party validation of d-Matrix's throughput claims to date.

CNBC reported that Microsoft is a d-Matrix investor through its M12 venture arm, and that the company has raised roughly $500 million to date at about a $2 billion valuation. Sheth told CNBC he sees a "$1 trillion market in the making" for inference compute and said he has no intention of selling the company. That backing, along with the shift toward agentic AI workloads that push inference volumes beyond what GPU-only infrastructure was built for, forms the demand backdrop for Corsair's production ramp.

Adopting a Corsair-like device means designing for heterogeneous rack-level orchestration: co-placing GPU-based prefill with Corsair-based decode, and running software that supports speculative decoding, sharding, or operator offload. d-Matrix's own materials describe this disaggregated prefill/decode pattern, already being explored elsewhere in inference research. Expect added system-level testing burden and a need for custom schedulers or runtime integration to realize advertised latency gains, rather than a drop-in replacement for existing GPU fleets.

Independent, reproducible benchmarks beyond vendor materials and the Gimlet Labs test remain the key gap: watch for third-party latency and throughput results on representative stacks (for example Llama-70B inference at scale), details on Corsair's SRAM capacity and chiplet sizing, cloud-provider pricing and availability timelines, and whether major model-serving toolchains add native support for disaggregated prefill/decode pipelines.