Decision tree:

1. What’s the target context length?

• ≤ 32K: pure attention is fine. KV cache is manageable; FlashAttention efficient. Default to it.
• 32K-128K: still attention-friendly. GQA + FlashAttention 3 handles this comfortably on H100/B200.
• 128K-1M: hybrid (Jamba, Samba, Granite 4) becomes very competitive. Attention is feasible but expensive.
• 1M+: hybrid or pure SSM clearly wins. Pure attention is prohibitively expensive.

2. What’s the deployment target?

• Datacenter GPU (H100+): both options work. Pick by quality/cost.
• Edge / mobile: SSM or hybrid. Fixed state dramatically simpler than growing KV cache.
• Browser / WASM: SSM (or attention with very small context). Memory is the constraint.
• Multi-tenant SaaS: attention’s KV-cache-per-request can dominate; hybrid is friendlier.

3. What’s the quality requirement?

• ”State of the art”: pure attention. Hybrid is 1-3% behind on most benchmarks.
• ”Production-ready chat / docs”: hybrid is comparable. Quality difference is small.
• ”Specific domain fine-tune”: either works. Domain adaptation usually swamps architecture choice.

4. Dataset / training data availability?

• Pretrain from scratch: hybrid models exist as open weights (Jamba, Granite 4); use those as starting points.
• Fine-tune existing model: attention (Llama, Qwen) has 10× more ecosystem support for LoRA/QLoRA. SSM fine-tuning is younger.
• Use API only: doesn’t matter — call the API.

Concrete recommendations:

• Building a chat app with 32K context: Llama 3 / Qwen 2.5 (attention). Maturity + tooling wins.
• Document understanding at 1M+ context: Jamba or Granite 4 (hybrid). Attention infeasible.
• On-device assistant: Falcon-Mamba 7B or similar SSM. Memory budget matters.
• State-of-the-art research model: attention (you’re competing at frontier; tooling matters).

The honest version: most projects in 2025 default to attention because the ecosystem is more mature. Hybrid/SSM is for specific use cases where context length or inference cost is the binding constraint.

Why these tasks expose the weakness:

The common thread: tasks that require RETRIEVING SPECIFIC INFORMATION from earlier in the context, rather than producing fluent continuation.

Long-context recall (Needle in a Haystack):

Task: 100K tokens of context, a specific fact buried in the middle, query at the end. The model must locate and reproduce the fact.

Why SSM fails: the state’s fixed compression means the specific fact (a few tokens of information) gets averaged with 100K other tokens. The signal is lost in the noise.

Why attention works: every token’s K, V is stored exactly. The query at the end can DIRECTLY ATTEND to the buried fact’s keys, retrieving its value precisely.

Multi-hop reasoning:

Task: requires retrieving multiple distinct pieces of information and combining them.

Why SSM fails: each retrieval pass through the state is lossy; combining multiple retrievals compounds the loss. By hop 3-4, the information is too degraded.

Why attention works: each “hop” can be a fresh attention lookup with high fidelity. Multi-hop reasoning compounds reliably.

Code generation:

Task: write a function that uses utilities defined earlier in the file (or in the context). The model must reference specific symbols by exact name.

Why SSM fails: symbol names are arbitrary tokens with no semantic shorthand. The state can’t compress “function called calculate_metric” into anything shorter without losing the exact name. When asked to generate code that USES it, the recall is fuzzy.

Why attention works: tokens are stored exactly in the KV cache. When generating a call to the function, the model attends back to its definition and gets the exact name.

How hybrids close the gap:

The hybrid layers have a key insight: precise recall is needed ONLY occasionally. Most of the FFN/Mamba layers do bulk processing; the few attention layers handle the recall-needs.

For Needle in a Haystack: even one attention layer with access to the full KV cache can find the needle.

For multi-hop reasoning: a few attention layers can handle each hop precisely; the Mamba layers in between can do the “reasoning” computation cheaply.

For code generation: attention layers retrieve symbol names exactly; Mamba layers compose the code.

Empirically: hybrid Mamba+attention models recover 80-95% of the attention-only quality on these specific tasks, at substantially lower compute. Pure Mamba is ~50-70% — usable but noticeably worse.

The “few attention layers” approach is the pragmatic resolution to a fundamental architectural trade-off. It’s likely the long-term direction.

Base prediction (most likely):

Frontier 2027 models will be predominantly attention-based but with significant SSM-inspired elements. The specific incarnation:

• Most layers: attention with sliding-window or global+local mix, plus FlashAttention 4+ efficiency.
• Some layers: pure Mamba or RWKV-style linear-time layers, used for long-distance integration.
• Sublayer-level mixing: even within a transformer block, some heads might use attention and others Mamba.
• Context length: 1-10M tokens routine. Some 100M-token experiments.

This is the “hybrid path” — attention dominant but with SSM-style components mixed in for efficiency.

Alternative path: full hybrid takeover.

If Jamba 4 or similar hits a breakthrough on quality, the frontier might shift to Mamba-dominant + attention layers. Probability: 30%.

Alternative path: attention reigns.

If hardware improvements (B200 → R100 → …) make attention’s O(N²) acceptable at 10M+ context, the frontier might never adopt SSMs. Probability: 25%.

Alternative path: something else.

A genuinely new architecture appears (e.g., a Diffusion-based language model from research, or a successor to both attention and SSMs). Probability: 15%. The last 3 years of “transformers will be replaced” haven’t yet delivered on this, but it’s possible.

What could change the prediction:

• A pure-SSM frontier model proving its quality advantage at scale (e.g., DeepSeek or Anthropic shipping a pure-Mamba 70B that matches GPT-5).
• A breakthrough in attention efficiency (FlashAttention 5+ with sub-quadratic scaling in practice).
• Custom silicon for SSMs (Groq-like LPUs optimised for recurrent computation).
• A “language modeling at 1M+ context” task that becomes economically critical and forces the architecture shift.

The honest investor advice:

Bet on hybrid architectures. They’re the path of least resistance — incremental adoption from existing attention models, with concrete cost wins. Pure SSM bets are higher-variance: bigger upside if they work, but they’ve underperformed expectations for two years now.

The honest practitioner advice:

Default to attention models (Llama, Qwen, etc.) for any new project. Switch to hybrid (Jamba, Granite 4) ONLY when long context or inference cost is the binding constraint. Don’t bet on pure SSMs in production until they prove themselves at scale.