Trellis Introduces RadixAttention KV Prefix Cache

According to the Trellis blog post, Trellis introduced RadixAttention, a radix-tree-based KV cache intended to accelerate the prefill phase of transformer inference. The post states Trellis targets deployments on users' existing hardware, including laptops, workstations and servers, and focuses this optimisation on chat-style and agentic LLM sessions where request sequences share common prefixes.

Per the Trellis blog post, the implementation treats keys and values as append-only during autoregressive generation and stores shared prompt prefixes in a radix tree, which collapses common substrings (for example, "hello my name is ") into single entries to reduce duplicated storage of suffixes like names. The post frames this as a precompute-and-reuse strategy for K/V matrices across requests that share prefixes.

Radix trees are a compact prefix representation that can cut both memory footprint and the amount of projection work needed during prefill when many sessions reuse similar prompt templates. For LLM inference stacks, this tradeoff typically lowers peak memory and prefill latency at the cost of maintaining an indexed prefix structure and handling cache lookups.

Many on-device and low-resource inference deployments face the same prefill cost; techniques that deduplicate K/V across sessions are therefore broadly useful to reduce compute and memory pressure for chat and agentic workloads.

Observers should watch for published benchmark numbers, broader OSS adoption of radix-based KV caches, and comparisons versus other caching strategies (sharded caches, chunked K/V, or token-level compression) to quantify real-world latency and memory benefits.