Qualcomm Unveils HBC Near-Memory AI Architecture: 6x Energy Efficiency Over HBM, Changing the Game for AI Inference

The "memory wall" in AI inference has long been a pain point for the industry. Compute power grows fast, but bandwidth can't keep up—no matter how powerful the GPU, it still has to wait for data.

HBM is the mainstream solution, but it's expensive, hot, and complex to package.

Qualcomm's answer yesterday? HBC. A near-memory computing architecture that packs AI accelerators directly beneath the memory stack, claiming

💡 What You Will Learn

Qualcomm Unveils HBC Near-Memory AI Architecture: 6x Energy Efficiency Over HBM, Changing the Game for AI Inference The "memory wall" in AI inference has long been a pain point for the industry. Comp

📜 Table of Contents

• How Bad Is the Memory Wall?
• What HBC Does
• Roadmap: From LPDDR5X to Two Generations of HBC
• One Blind Spot Left Unmentioned
• My Take: The Real Killer Feature Isn't Performance—It's Cost
• Let's Discuss in the Comments

Qualcomm Unveils HBC Near-Memory AI Architecture: 6x Energy Efficiency Over HBM, AI Inference Is About to Change

The "memory wall" in AI inference has long been a pain point for the industry. Compute power grows fast, but bandwidth can't keep up—no matter how powerful the GPU, it still has to wait for data.

HBM is the mainstream solution, but it's expensive, hot, and complex to package.

Qualcomm's answer yesterday? HBC. A near-memory computing architecture that stuffs an AI accelerator directly beneath a memory stack, claiming 6x the bandwidth efficiency of HBM and 200x the capacity of on-chip SRAM.

If this actually ships, the rules of the game for AI accelerators might just get rewritten.

How Bad Is the Memory Wall?

Let's first pin down the problem.

During large model inference, most of the time isn't spent computing—it's spent "waiting for data." No matter how fast the compute unit runs, if data takes hundreds of cycles to move from memory, overall performance is bottlenecked. That's the memory wall.

HBM is currently the best bandwidth solution—it connects multiple DRAM stacks to the compute chip via a silicon interposer, delivering TB/s-level bandwidth. But the trade-offs are obvious:

HBM is expensive: a single HBM3E stack costs hundreds of dollars HBM runs hot: high bandwidth means high power consumption HBM packaging is complex: it requires advanced packaging like CoWoS and silicon interposers, and capacity is constrained

Qualcomm's HBC essentially aims to bypass HBM's bottlenecks with a cheaper, more power-efficient approach.

What HBC Does

The idea is brutally simple.

The conventional approach separates the accelerator chip and HBM memory, connecting them through an interposer. HBC does this: it pulls the AI accelerator out of the SoC and places it directly beneath the LPDDR memory stack.

The two are connected vertically through through-silicon vias (TSVs), making the data path nearly negligible. No expensive HBM, no complex silicon interposer—standard packaging can house multiple HBC stacks in a single compute device.

Tony Pialis, head of Qualcomm's data center business, put it bluntly:

"We separated the AI accelerator from the XPU and placed it directly under the DRAM stack. That means we get SRAM-level performance advantages while retaining the density and capacity benefits of stacked memory. HBM's connection congestion is gone. Lower power, less heat, and we eliminated the costly silicon interposer."

The numbers are striking: 6x better bandwidth efficiency than HBM, and over 200x the capacity of on-chip SRAM.

Roadmap: From LPDDR5X to Two Generations of HBC

Qualcomm also shared a timeline:

• AI200 — Based on LPDDR5X, delivering 43TB of memory per rack (by end of this year)
• AI250 — First-gen HBC, 18x the bandwidth of AI200
• AI300 — Second-gen HBC, 54x the bandwidth of AI250

Note the jumps—from LPDDR5X to first-gen HBC is an 18x leap, and to second-gen is a staggering 54x. That means AI250 alone could handle inference tasks that today require multiple H100s interconnected.

One Blind Spot Left Unmentioned

For all the benefits, Qualcomm is tight-lipped on one detail: what exactly is the accelerator inside HBC?

Is it a dedicated Transformer inference engine? A general-purpose tensor core array? Or some kind of AI training preprocessing logic?

Theoretically, any of these could work, but Qualcomm isn't giving an answer.

This reminds me of how every DRAM maker has dabbled in "near-memory computing" experiments that never went mainstream—GUC's recent DRAM-on-Logic technology is similar, stacking 1-4 layers of DRAM on logic and claiming 5 TB/s bandwidth.

The technical path isn't new, but whether it can scale to production is what matters.

My Take: The Real Killer Feature Isn't Performance—It's Cost

HBM's current problem is: it's great, but too expensive. In an H100's BOM, HBM takes up a significant chunk. CoWoS packaging capacity has been a long-term bottleneck, even NVIDIA has to fight for it.

If Qualcomm's HBC can deliver HBM-like bandwidth using standard packaging and eliminate the silicon interposer, its biggest value isn't "how fast it runs"—it's lowering the hardware barrier for AI inference.

Especially for inference scenarios—once a large model is deployed, inference cost is the key to mass adoption. If HBC can cut inference costs in half with a cheaper packaging solution, its market impact could be bigger than the "18x bandwidth" number.

Of course, that's assuming it ships. Qualcomm has a history of over-promising in the data center space. Whether this time is different depends on AI250's actual product next year.

Let's Discuss in the Comments

Is Qualcomm taking a slice of HBM's pie, or is this another "Qualcomm hype"? Do you think HBC can really kill HBM, or will it remain a lab sample? Drop your take in the comments.