State space model (SSM)

A state space model (SSM) is a type of AI model that reads text or other data step by step and keeps a fixed-size running summary. A transformer, the design behind most language models, instead keeps every earlier word and compares each new one with all of them. That makes SSMs cheaper on long inputs, such as contracts or chat histories, but details can drop out of the summary.

Where does the idea come from?

State space models are a classic tool in control theory and other sciences: a system has a hidden state that changes with every input, and the output is read from that state. S4 (Albert Gu, Karan Goel and Christopher Ré, 2021) made the idea work in deep learning for long sequences. It solved Path-X, a test with sequences of 16,384 steps on which all earlier work had failed, and generated 60 times faster than a transformer in the paper's tests. Earlier SSMs still lagged behind attention on language. Mamba (Gu and Tri Dao, December 2023) closed that gap: Mamba-3B beat transformers of the same size.

How does it work?

Simplified, one step of an SSM looks like this:

1
h_t = A · h_(t-1) + B · x_t
2
y_t = C · h_t

• x_t is the input at step t, for example a token, h_t the hidden state after that step and y_t the output.
• A sets how much of the old state is kept, B how much of the new input is written into it, and C what is read out.

S4 starts from the continuous version, a differential equation from control theory, and turns it into these discrete steps; A, B and C are learned in training. In S4 they are the same for every step, so training can process a whole sequence at once as one long convolution. When the model generates text, it runs step by step, and each new token costs the same work and memory, however long the text already is.

Mamba made the parameters depend on the current token, so the model decides per token what to keep and what to forget; its authors call this selection. That rules out the convolution, so Mamba uses a parallel algorithm tuned to the GPU instead. The Mamba entry describes that line of models, up to Mamba-3 (March 2026).

How does it compare with a transformer?

Attention compares every token with all earlier ones and keeps their keys and values in the KV cache. The Mamba-3 paper sums up the cost as "quadratic compute and linear memory"; an SSM needs linear compute and constant memory. In the Mamba paper, Mamba generated text with five times the throughput of transformers. At 1.5 billion parameters, Mamba-3 averaged 56.4 on seven language tasks, against 55.4 for a transformer of the same size.

The weak spot is exact recall. The Mamba paper describes the trade-off: attention does not compress its context, which makes it effective but slow, while a model with a fixed state is only as good as what fits in it. The Mamba-3 authors found their model competitive on question answering, but it struggled to extract information from semi-structured and unstructured documents. They predict that such layers will mostly be used in hybrids that keep some attention layers. NVIDIA's Nemotron-H (April 2025) replaces most attention layers with Mamba layers and reports up to three times faster inference than similar open transformer models. IBM's Granite 4.0 hybrids (October 2025) combine Mamba-2 and transformer blocks in a 9:1 ratio.

Example

H Company's Holotron-12B (March 2026) is a model for computer-use agents, which operate a browser or desktop by looking at the screen. It is built on NVIDIA's Nemotron-Nano-2 VL, a hybrid of SSM and attention layers. According to H Company, the SSM layers store a constant state per layer and per sequence instead of a growing KV cache, which leaves room for more requests on the same GPU. On the WebVoyager benchmark, with 100 agents working at once on a single H100 GPU, Holotron-12B reached over twice the throughput of H Company's earlier Holo2-8B.