Reflection Releases Beam: An Open 501B-Parameter MoE Model Under Apache 2.0

Reflection, a new US lab, has published its first model, Beam, on Hugging Face. It is a 501B-parameter MoE with 23B parameters active per token, text input, a 1M-token context and an Apache 2.0 license. It is built for coding and agentic tasks, and the target audience is clear: teams that want a frontier-scale model on their own hardware or in their own cloud, with the option to fine-tune it for their needs.

The release ships with Reflection's own harness written in Rust, open-sourced like the model itself. Reflection and Hugging Face jointly prepared recipes for inference, SFT and RL, and transformers, vLLM and SGLang support is available from day zero. For full-parameter RL, they suggest running it through Miles.

Where Beam Saves and Where the Memory Bill Comes In

It is the classic sparse MoE deal: you compute like a small model but pay for memory like a large one. Beam has 52 layers. The first three are dense, and the remaining 49 are replaced with blocks of 256 experts, with 8 routed per token. Each expert is a SwiGLU MLP with an intermediate size of 2560 and about 39M parameters, and together they hold 98.4% of the weights. There is no shared expert. The router follows the DeepSeek-V3 scheme with group limits: sigmoid scores, 8 groups of 32 experts, the 4 best groups, then top-8 within them. Expert outputs are normalized and scaled by 2.5.

In BF16 the checkpoint weighs about 1 TB. NVFP4 for Blackwell squeezes it to roughly 280 GB, with some parameters kept in BF16 for accuracy. FP8 is available for Hopper and Ampere. Two B200s with 180 GB each can technically hold the weights but leave no room for the KV-cache, so the authors recommend four B200s (720 GB). An 8×B200 node, two B300s with 288 GB each, and even 8×RTX Pro 6000 also work. On H200, you need at least 16 cards: with 8 GPUs, the weights take 139 GB per card, which leaves no room on a 141 GB card.

Hybrid Attention and Unusual Residuals

The cache is the second major cost, and here Beam looks well engineered. Of the 52 layers, 43 use a 512-token sliding window with RoPE, and only 9 (every sixth layer plus the last) attend to the full context without positional encoding. Instead, the query is multiplied by a slowly growing scaler, (1 + (pos + 1) / 2048) ** 0.1, which acts as a temperature and keeps softmax from smearing out over long distances. The cache grows by 36 KiB per token instead of 208 KiB. At the full 1M context, that is 36.08 GiB versus 208 GiB, and at 128K it is 4.6 GiB versus 26 GiB. The figures are per sequence in BF16.

Attention uses GQA with 48 query heads and 8 KV heads, RMSNorm on queries and keys, plus a learnable sigmoid gate for each head channel. The most unusual detail is the residual stream. At step k, the sublayer output is added with weight 1/sqrt(k), while the accumulated stream is multiplied by sqrt((k-1)/k). The squared weights sum to one, the stream scale stays constant, and after 104 additions the embedding and each sublayer contribute equally. Logits pass through a tanh softcap at ±30 in FP32.

What the Benchmarks Show and How to Run It

The figures come from the Reflection team, for BF16 / FP8 / NVFP4:

• SWE-bench Verified: 81.6% / 80.2% / 80.7%
• SWE-bench Pro v1: 66.1% / 65.9% / 65.6%
• SWE-bench Pro v2-Hard: 76.5% / 78.4% / 78.6%
• SWE-bench Multilingual: 78.3% / 78.1% / 77.5%
• Terminal Bench v2.1: 80.1% / 78.9% / 78.2%
• DeepSWE v1.1: 43.3% / 43.0% / 42.7%
• AIME 2026: 97.7% / 98.0% / 97.5%
• GPQA Diamond: 90.6% / 90.3% / 89.3%
• LiveCodeBench: 88.3% / 88.1% / 87.4%
• HLE without tools: 36.0% / 36.0% / 34.8%; with tools: 51.1% across all three
• MCP Atlas: 80.8% / 81.5% / 80.5%
• BrowseComp with context management: 76.8% / 76.7% / 75.6%
• DeepSearchQA: 80.5% / 79.6% / 81.1%
• LongBench v2: 67.5% / 67.6% / 67.2%
• IFBench: 79.6% / 80.7% / 79.2%
• AA-LCR: 78.2% / 79.4% / 78.7%

The main takeaway from the table: moving to NVFP4 costs a fraction of a percentage point on average, while a weight footprint half the size makes self-hosting realistic.

The model has five reasoning levels (low, medium by default, high, xhigh, max). The preserve_thinking option keeps reasoning across turns, and tool calling uses a separate XML-like grammar. The tokenizer includes a response template, and parse_response separates reasoning, answer and tool calls without regexes, casting arguments to the types in the tool schemas. For speedup, there is DFlash2 speculative decoding: a draft model proposes up to 15 tokens, and Beam verifies them in a single pass.

In practice, launching looks like this. In vLLM, BF16 runs with tensor-parallel-size 16 on two 8×H200 nodes, while in SGLang NVFP4 fits on 4 B300s with a context of 1 048 576. In transformers (version 5.20.1 and above), expert parallelism works with experts placed one per device, while attention, dense layers and embeddings are replicated (8B parameters, 16 GB). The number of devices must divide 256. For those without Blackwell, serverless Inference Providers are available. Reflection will also hand out $10 000 in Hub credits for the best projects: demos, quantizations, synthetic datasets, RL environments or benchmarks.