Search

DeepSeek V4-Flash: Open-Weight, Million-Token Context

On May 16, DeepSeek quietly dropped a preview of the DeepSeek-V4 series on HuggingFace. No keynote, no press tour, no viral demo. Just a README, a model card, and a set of benchmarks that should make every frontier lab take notice. The headline? DeepSeek-V4-Flash - an open-weight, MIT-licensed model with a 1 million token context window, 284 billion total parameters, and only 13 billion active at any given time. That makes it the first open-source model to offer million-token context at a parameter scale that can actually run locally.

What DeepSeek Actually Released

The V4 series ships in two variants:

  • DeepSeek-V4-Flash - 284B total params, 13B activated, 1M context, FP4/FP8 mixed precision. MIT license.
  • DeepSeek-V4-Pro - 1.6T total params, 49B activated, 1M context, FP4/FP8 mixed precision. Also MIT license.

Both use a Mixture-of-Experts (MoE) architecture with what DeepSeek calls "Hybrid Attention" - combining Compressed Sparse Attention and Heavily Compressed Attention to bring the KV cache down to roughly 10% of what you'd expect from a dense model at this scale. In practice, that means the 1M context window is not just a spec-sheet number. It's usable.

The Pro variant is the frontier-class workhorse. The Flash variant is the one that changes the game for engineers who want to run models locally. At 13B active parameters, it sits in the same inference-efficiency neighborhood as Llama 4 70B or Qwen3.6 27B - models that already run comfortably on consumer GPUs. But DeepSeek's benchmarks suggest Flash-Max (the highest reasoning mode) punches well above its weight.

The Benchmark Story

DeepSeek published head-to-head numbers against Claude Opus 4.6 Max, GPT-5.4 xHigh, and Gemini 3.1 Pro High. The results are selective - every vendor cherry-picks - but the pattern is revealing:

  • LiveCodeBench: V4-Pro Max scores 93.5 versus Opus 4.6 Max at 88.8. That's a decisive coding win.
  • Codeforces rating: V4-Pro Max hits 3206, the highest in the table.
  • SWE Verified: V4-Pro Max matches Opus at 80.6.
  • MMLU-Pro: V4-Pro Max trails Gemini-3.1-Pro (87.5 vs 91.0) but edges out GPT-5.4.

For the Flash variant specifically, the gap to Pro is narrower than the parameter counts suggest. Flash-Max scores 91.6 on LiveCodeBench (just 1.9 points behind Pro) and 3052 on Codeforces. On long-context tasks, Flash-Max hits 78.7 MRCR at 1M tokens - not class-leading, but entirely respectable for a model with 13B active parameters.

The takeaway: DeepSeek V4-Pro Max is competitive with the best closed models on coding and reasoning. V4-Flash Max gets surprisingly close at a fraction of the inference cost.

The Context Window Arms Race Just Went Open-Source

Until this week, the million-token club had exactly two members: Gemini Ultra 2 (9.2 LMRank score, $10/M input) and Grok 4.3 (8.5 score, $1.25/M input). Both are proprietary APIs. You can't download the weights, can't run them offline, can't modify them, and can't inspect their activations.

DeepSeek-V4-Flash changes that equation completely. It's an open-weight model with a 1M context window. Under an MIT license. That means:

  • Local deployment - Run it on your own hardware without API rate limits or data egress concerns.
  • Steering and interpretability - Access to internal activations enables techniques like steering vectors, which Sean Goedecke's recent analysis argues are finally practical thanks to models like this.
  • Fine-tuning - Adapt the model for domain-specific retrieval or long-document workflows without relying on a provider's fine-tuning pipeline.

This is not a theoretical advantage. For legal discovery, biomedical literature review, financial document analysis, and codebase-scale retrieval, a 1M context window changes what tasks are even possible. Until now, those workloads required either Gemini Ultra 2 or Grok 4.3 - both excellent, both closed. DeepSeek just made the open-source path viable.

How It Fits the LMRank Leaderboard

DeepSeek V4 already sits at rank 7 on LMRank with a 9.0 score and $0.50 per million input tokens. It's the highest-ranked open-weight general model on the board, trailing only Claude Opus 4.7, Claude Opus 4.5, GPT-5.5, GPT-5, and Gemini Ultra 2.

The new V4-Flash and V4-Pro preview variants extend that family with two critical upgrades: the 1M context window and the MoE efficiency gains. If DeepSeek's own benchmarks hold up under independent evaluation, V4-Pro Max would likely score in the 9.1–9.3 range - right in the mix with GPT-5 and Gemini Ultra 2. V4-Flash Max might land around 8.7–8.9, competing with Llama 4 405B and Qwen3.6 Max Preview.

Pricing is still unconfirmed for the API versions, but if DeepSeek maintains its historical pricing discipline, expect V4-Flash to land under $1 per million input tokens - making it the cheapest million-token model by an order of magnitude.

Scores at a glance: DeepSeek V4 (9.0) · Gemini Ultra 2 (9.2) · Grok 4.3 (8.5) · Llama 4 405B (8.7). See full pricing, context windows, and rankings on the LMRank leaderboard.

The Architecture Is the Story

DeepSeek-V4 isn't just bigger. It's structurally different from the dense models that dominate the frontier. The "Hybrid Attention" mechanism is the key - it uses Compressed Sparse Attention for most tokens and Heavily Compressed Attention for long-context positions, collapsing the KV cache to roughly 10% of what DeepSeek-V3.2 required. At 1M tokens, that efficiency difference is the gap between "fits on an A100" and "needs a DGX cluster."

The MoE setup also matters. With 13B active parameters out of 284B total, Flash only loads the experts it needs for a given token. That keeps inference latency low without sacrificing the model's expressive capacity. It's the same efficiency philosophy that made Qwen3.6 35B A3B (8.5 score, 35B total / 3B active) and Kimi K2.6 (8.4 score) so compelling for coding workflows. DeepSeek just applied it at a much larger scale.

What to Watch For

Preview releases are not production releases. Three questions need answers before V4-Flash earns its place on the LMRank leaderboard:

  • Independent benchmarks - DeepSeek's numbers are promising, but third-party evaluation on LMSYS, LiveBench, and LMRank's own creative tests will tell the real story.
  • Long-context accuracy - 1M tokens is meaningless if retrieval accuracy collapses past 200K. The MRCR and CorpusQA numbers look solid, but real-world needle-in-haystack tests are the true test.
  • API pricing and availability - Open weights are great for researchers. API pricing determines whether this model actually displaces Grok 4.3 and Gemini Ultra 2 for production workloads.

If those three checks come back positive, DeepSeek-V4-Flash becomes the default recommendation for anyone who needs long-context capability without writing a five-figure monthly check to a closed API. And V4-Pro becomes the first open-weight model that can legitimately claim parity with Claude Opus on code generation.

The Bottom Line

DeepSeek's V4 preview is the most significant open-source model release of 2026 so far. Not because it tops every benchmark - it doesn't - but because it expands what open weights can do into territory that was exclusively proprietary. A million-token context window, competitive coding performance, and MIT licensing add up to a genuine inflection point.

For the LMRank leaderboard, this means the gap between "open" and "frontier" is about to get even narrower. And for developers, it means the best long-context model for your next project might not require an API key at all.

At lmrank.com, we track live scores, pricing, and context windows for every major LLM. Follow DeepSeek V4's detail page for updates as independent benchmarks validate the new V4-Flash and V4-Pro variants.

Sources

See also: Longest-Context Models