[OCP in Focus] SK hynix Drives up to 5x Faster Time-to-First-Token with MARS; HBF Expands Capacity by 14x

To break free from the ongoing memory crunch, SK hynix is pushing the boundaries of AI infrastructure beyond conventional HBM solutions. At the OCP Global Summit 2026, SK hynix Senior Vice President Hoshik Kim introduced MARS (Memory-Centric AI Rack-Scale System), a disaggregated rack-scale architecture designed to seamlessly coordinate compute and memory resources.

Notably, Kim also spotlighted the progress of HBF (High Bandwidth Flash), targeting the industry’s pressing capacity constraints. Here is TrendForce’s on-site summary of his key remarks.

MARS: Boosting TTFT up to 5x to Solve Processor-Centric Cluster Bottlenecks

According to Kim, traditional processor-centric architectures face severe structural limitations in distributed computing clusters. These include resource imbalance—where idle memory on one server cannot be easily shared with another—as well as rigid scalability, where adding memory requires purchasing an entire new server node. Consequently, operators are forced into GPU over-provisioning, driving up TCO and creating system inefficiencies.

“By memory disaggregation, this decouples memory from traditional CPU/GPU server constraints, creating a dedicated, massive memory pool… we can actually move from a per-server memory headroom to a pool-level efficiency,” Kim said.   

As explained by Kim, MARS is a disaggregated rack-scale memory system that coordinates compute and memory as a shared infrastructure. “Now we are expanding this concept of the memory tray-level memory pool as a rack-scale memory pool.”

Highlighting the real-world performance gains enabled by MARS, Kim noted: “We applied this one into a multi-turn AI agentic AI inferencing service. And then by providing this system, we can actually consume and store this exploding KV cache through this multi-turn AI agents into a shared memory system without exhausting GPU memory and storage systems.”  

“So if you look at this graph, we have a very good 3 to 5 times improvement in TTFT (Time To First Token) over recomputation, which is quite impressive, especially when you talk about this tail latency, tail latency is dramatically reduced.”

HBF: Delivering 14x HBM Capacity with Sampling Set for 2028

SK hynix also elaborated on its progress with High Bandwidth Flash (HBF), highlighting its key performance strengths.

“Basically, HBM is a good memory… So HBF expands this HBM-class bandwidth to rack-scale capability/capacity… HBF is typically half the bandwidth of HBM, but more like 14 times higher capacity than HBM,” Kim said.

He confirmed that product development is well on track. “Recently in July, we released OCP HBF spec version 0.7… expecting to release spec version 1.0 early next year, and currently developing HBF concept prototype, and we’ll sample HBF Cube available early 2028.”

Read more

• SK hynix’s OCP Global Summit 2026 Keynote Highlights
• [News] SK hynix, SanDisk Debut HBF Standard to Challenge AI Memory Bottlenecks with Google, Tenstorrent Support

(Photo credit: TrendForce)