Flash Just Got a Standard for Holding Model Weights. Google Helped Validate It.

Every inference engineer has run the same losing arithmetic. The model is bigger than the memory. The memory is the expensive part. High Bandwidth Memory is fast and small and priced accordingly, and the moment your weights exceed one node you start paying a networking tax nobody budgeted for.

This week in Santa Clara, two memory companies proposed a different answer: stop making HBM carry the bulk of the weights, and give it a neighbor.

Sandisk and SK hynix published the first technical specification for High Bandwidth Flash, a NAND-based memory tier designed to sit between HBM and SSDs, through the Open Compute Project. Sandisk announced from Milpitas on August 3. SK hynix announced from Seoul on August 4, the opening day of the Future of Memory and Storage conference. The consortium formed in February, following an initial standardization partnership between the two companies in August 2025.

Six months from working group to published standard is fast. What actually landed is more concrete than the phrase "first specification" usually implies.

The Spec Puts Numbers on the Table

This is not a whitepaper describing a direction. The first release defines the shape of the thing.

The UCIe choice is the piece most worth noticing. UCIe (Universal Chiplet Interconnect Express) is the existing open standard for wiring heterogeneous chiplets together, already supported across the industry. Building HBF on top of it rather than inventing a bespoke link means an HBF stack can, in principle, attach to a GPU, a CPU, or a custom accelerator without each vendor negotiating a private interface.

Three bandwidth grades matter for a similar reason. A single mandated speed forces every implementer into one cost structure. A graded ladder lets a hyperscaler buy the fast tier for latency-sensitive serving and the slow tier for cold weights, which is how memory hierarchies have always actually worked.

The Pitch Is Capacity, and the Physics Is Read-Mostly

HBF stacks NAND flash close to the compute die. NAND holds its contents when power is cut and needs none of the refresh power DRAM demands. Sandisk's implementation uses its CBA process, which bonds memory cells directly to CMOS circuitry.

The argument underneath rests on an observation about what inference does with model weights: it reads them, over and over, and almost never writes them. A pretrained checkpoint is a read-mostly asset. Paying HBM prices to store a read-mostly asset is, in this framing, a category error.

Sandisk acknowledges the tradeoffs plainly. HBF carries longer latency and larger page sizes than HBM. The two tiers are designed to coexist, with HBM covering the hot working set and write-intensive work while HBF holds bulk weights far closer to compute than any SSD could.

SK hynix has a name for the resulting architecture, and it built its FMS keynote around it. Executive Vice President Kim Chun-sung, head of solution development, and Vice President Kang Uk-song, head of next generation product planning, delivered a joint address titled "Orchestrating Efficient AI Infrastructure through Tiered Memory in the Era of Agentic AI."

Google Joining Is the Signal Worth Reading

The membership list tells you more than the document does.

Google and Tenstorrent joined the consortium partway through drafting, contributing to technology validation and specification development, with Sandisk and SK hynix as primary contributors. A memory standard written by two memory vendors is a proposal. A memory standard a hyperscaler and an accelerator designer helped validate is closer to a plan.

Google runs inference at a scale where memory cost per terabyte lands directly in the quarterly numbers, and it serves 22 billion tokens a minute. Tenstorrent designs its own accelerators and was itself the subject of a reported $10 billion approach from Qualcomm earlier this year.

Google's involvement goes past membership. On August 6, Google DeepMind Senior Staff Engineer Xiaoyu Ma joins SK hynix Vice President Lim Eui-cheol and Sandisk Vice President Rajeev Nagabhirava on a panel titled "Breaking the Memory Wall with High Bandwidth Flash," hosted by Coughlin Associates president Thomas Coughlin. Sandisk holds its own keynote on August 5 with chief revenue officer Jim Elliott, chief product officer Khurram Ismail, and chief technology officer Alper Ilkbahar.

The Evidence Is a Simulation, and the Conformance Test Does Not Exist

Here is where a careful reader should slow down.

Sandisk's headline performance claim comes from an internal simulation reading 8-bit pretrained weights from Llama 3.1 405B, in which the system-level gap between HBF and a hypothetical HBM configuration with unlimited capacity came in within 2.2%.

Unpack that and several things fall out. It is a simulation, not measured silicon. It models one kernel running at a time, not mixed workloads. It covers reads, not writes. The baseline is a hypothetical device that does not exist. Sandisk's own first-generation product targets, presented in July 2025, land at the spec's capacity ceiling but only about half its top bandwidth grade: 512 GB across a 16-die stack at 1.6 TB/s read bandwidth, with footprint, power, and stack height comparable to HBM4.

The specification has gaps of its own. As XenoSpectrum noted in its analysis, the release includes no version number, no conformance testing procedure, and no certification scheme. A common interface being defined does not guarantee that HBF from one vendor and an accelerator from another will interoperate. Pricing has not been disclosed, so nobody can compute total cost of ownership yet.

Then there is the timeline. As of August 2025, Sandisk had indicated targets of initial HBF memory samples in the second half of 2026 and initial samples of inference devices incorporating HBF in early 2027. SK hynix has projected that demand for composite memory including HBF grows around 2030. The standard arrived roughly four years ahead of the market it is meant to serve, which is either admirable foresight or a long time for a specification to sit unproven.

The least glamorous open problem is software. Capacity alone shortens nothing. Unless the runtime decides intelligently what belongs in HBM and what belongs in HBF, a bigger slow tier just relocates the stall. That scheduling problem has no standard, no reference implementation, and no owner.

Why This Lands Right Now

The timing is not coincidence. Open-weight models crossed a threshold this year that made memory capacity, rather than raw compute, the binding constraint on serving them.

• Moonshot's Kimi K3 needs more than 1.5 TB for weights before any cache allocation
• Alibaba's newest flagship carries 2.4 trillion parameters that almost nobody can run
• Practitioners are already finding cases where the GPU with more memory beats the faster GPU, because it avoids crossing a node boundary
• DRAM pricing has climbed hard enough that a memory company staged China's biggest IPO since 2010

A tier that trades latency for a large jump in near-compute capacity arrives into a market that has been asking for exactly that.

SK hynix is pressing the advantage on adjacent fronts too. At the same conference it revealed its tenth-generation 375-layer 4D NAND for the first time, claiming a 2.5x improvement in performance per watt over the previous generation, with mass production of enterprise SSDs built on it targeted for early next year.

The Bottom Line

The number to watch is not 512GB and it is not 3.0 TB/s. Both are ceilings, and ceilings have a way of becoming asterisks.

The number to watch is how many independent implementations talk to each other when samples ship in the back half of this year. A specification published without a conformance test is an invitation, not a guarantee. Every open standard in computing has faced the same fork: it becomes the boring layer nobody thinks about, or it becomes a document three companies interpreted three different ways.

Sandisk and SK hynix have spent six months arguing that the memory wall in AI inference is a capacity problem wearing a bandwidth costume. Picking UCIe instead of a proprietary link suggests they know the argument only works if other people can join it. Nobody will know whether they were right until somebody outside those two companies plugs one in.

Sources

• SK hynix and Sandisk Begin Global Standardization of Next-Generation Memory HBF (Aug 2025)
• Sandisk HBF Fact Sheet (2025)
• High Bandwidth Flash Unlocks Massive Model Storage (IEEE Spectrum, 2025)
• GPUs could explode to multiple TB with new storage-inspired memory tech (Jul 30, 2026)
• Sandisk and SK hynix Advance Global Standardization of High Bandwidth Flash (Press Release) (Aug 3, 2026)
• SK hynix Unveils First HBF Standard Specifications with Sandisk, Presenting AI Memory Solutions at FMS 2026 (Aug 4, 2026)
• Sandisk and SK hynix Publish First OCP Specification for HBF (XenoSpectrum, Aug 4, 2026)
• Sandisk and SK hynix Advance Global Standardization of High Bandwidth Flash (TechPowerUp, Aug 2026)
• FMS 2026: Sandisk and SK hynix Advance Global Standardization of High Bandwidth Flash (StorageNewsletter, Aug 5, 2026)
• OCP Semi-Private Workstreams (2026)