SK Hynix integrates near-memory dequantization architecture into custom HBM for LLM inference.

Researchers from SK hynix have published a technical paper introducing StreamDQ, a novel approach to accelerating AI inference. The paper, titled "StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Acceleration," proposes a lightweight architectural enhancement that enables on-the-fly dequantization in the memory subsystem for high-throughput, large-batch LLM inference.

The StreamDQ architecture delivers significant performance improvements, achieving up to 7.08× speedup and 90.23% lower energy consumption for mixed-precision GEMM operations. By performing dequantization near memory rather than in the compute core, the approach addresses a key bottleneck in quantized LLM inference, where bandwidth-constrained memory operations have traditionally limited throughput.