Mixture of Recursions

One block, reused. A router decides how many times each token passes through it, so easy tokens leave early and hard ones keep going — the same quality for a fraction of the compute.

← All demos View source
MoR — forward passwatch a sentence walk the network
INPUT TOKENS ROUTER depth per token — SHARED RECURSION BLOCK RMSNorm pre-norm Grouped-query attention RoPE · FlashAttention SwiGLU feed-forward gated MLP PASS 0 OF 0 weights shared across every pass KV CACHE recursive reuse OUTPUT RECURSE — SAME WEIGHTS, AGAIN
0block passes used
0a fixed 12-layer stack would use
0%fewer FLOPs on this sentence
~3×fewer unique parameters
Runs automatically · depths shown are the routing decision per token

Why it is cheaper

A standard decoder runs every token through every layer. MoR routes per token, so "the" can leave after one pass while a rare word keeps recursing.

One block, many depths

Depth comes from reuse, not from more parameters. Cross-recursion weight sharing is what buys roughly three times fewer unique parameters.

Recursive KV caching

Keys and values from earlier passes are kept and reused, so extra depth does not mean recomputing attention from scratch each time.

Measured result

Up to 2× faster inference and 40–60% fewer FLOPs at comparable modelling quality — a from-scratch PyTorch implementation of arXiv:2507.10524.