Skymizer Announces HTX301 Single-Card 700B Inference

Per Skymizer's announcement and a PR Newswire release, Skymizer unveiled the HTX301 inference chip and the HyperThought hardware/software platform, presented as a reference design for on-prem inference. The company and the PR state a single PCIe card populated with six HTX301 chips and 384 GB of memory can execute 700B-parameter model inference at roughly 240W per card (PR Newswire; Skymizer announcement). Press coverage from TechRadar and Wccftech reports additional vendor claims about processor node and memory choices, specifically use of 28 nm silicon and LPDDR4/LPDDR5 rather than HBM or GDDR (TechRadar; Wccftech).

Per Skymizer and reporting, the HyperThought architecture emphasises a decode-first approach and disaggregates prefill and decode workloads, pairing purpose-built LPU IP with a software orchestration layer. Skymizer's materials claim compression techniques for model weights and KV cache that outperform open-source baselines, with quoted ranges such as 9% to 17.8% weight-compression improvement versus llama.cpp and KV-cache perplexity loss in the low-percentage range (Wccftech; Skymizer announcement). Tech outlets cite vendor-published microbenchmarks including figures like 30 tokens/second at 0.5 TOPS and bandwidth claims around 100 GB/s (TechRadar; Wccftech). These numerical claims in the announcement and press coverage are vendor-sourced or drawn from early reporting; independent third-party benchmarks are not present in the scraped coverage.

What is reported is primarily company-provided specification and selected microbenchmarks, amplified by tech press coverage (PR Newswire; Skymizer site; TechRadar; Wccftech). Reporting does not include independent third-party benchmarks, long-duration reliability data, denoised latency distributions for large-vocabulary tasks, or details on software compatibility with major model runtimes beyond vendor descriptions. For high-stakes claims, single-card 700B inference at 240W and memory/compression trade-offs, the sources are explicit that these are vendor announcements (Skymizer announcement; PR Newswire). The announcement includes executive quotations such as, "Inference has become the dominant AI workload, and infrastructure needs to reflect that reality," attributed to William Wei, Chief Marketing Officer (Skymizer announcement; PR Newswire).

public reporting frames the HTX301 announcement as an attempt to reduce reliance on multi-GPU clusters, NVLink/NVSwitch interconnects, and high-bandwidth memories for very large LLM inference. Companies promoting decode-optimised silicon and compressed KV/weight representations argue such designs can lower end-to-end power and infrastructure cost for certain inference workloads. Industry context: similar trade-offs have appeared in prior accelerator projects that trade peak TOPS for efficiency via compression, reduced-precision math, and workload-specific pipelines. The claim set here combines lower process-node silicon with heavy compression and orchestration, a combination that, if validated, would change some deployment economics for on-prem enterprise inference but would require independent validation on realistic workloads and model families.

observers should look for independent benchmarks from neutral labs or community projects comparing the HTX301 card to contemporary GPU accelerators on key axes: end-to-end inference latency, real-world throughput on common LLM families, memory-residency behaviour for KV cache, quantization and compression-induced accuracy regressions, and sustained power under production workloads. Additional signals include details on software ecosystem support (runtimes, frameworks, tokenizer compatibility), availability of hardware samples for partners, and vendor-provided reproducible benchmark artifacts. Media coverage and vendor materials to date focus on specifications and microbenchmarks; the transition to production adoption will require ecosystem integration and stability data (TechRadar; Wccftech; PR Newswire).

Overall, the announcement is notable for its combination of claims: single-card support for 700B-parameter models, use of older 28 nm process technology, and an emphasis on compression and decode-first orchestration. Those claims are currently vendor-originated and await independent verification before the broader AI infrastructure community can assess practical performance and cost trade-offs.