Posts Tagged "sparse-attention"

At each decoding step, a sparse-attention model reads a sliver of a session's history. Adaptive HiSparse keeps the rest in host memory, lends it GPU space when there is room, and takes the space back when new requests need it. On DeepSeek-V4.1-Flash at 500K tokens of context, that is 20 to 27% more output from the same GPUs at high load.