DEV Community

Cover image for K2-Horizon Opened Everything: 375B Open Weights, Data and All
Paul Lovelace
Paul Lovelace

Posted on Edited on Originally published at toolscout-e2i.pages.dev

K2-Horizon Opened Everything: 375B Open Weights, Data and All

Most "open" model releases give you the weights and wish you luck. IFM just did something I haven't seen at this scale: they opened the weights, the training data, the intermediate checkpoints, the training logs, the infrastructure code, and the pipelines that built the data. All of it. That's the story here — not another model, but a release that treats reproducibility as the product.

On November 16, 2026, IFM (Abu Dhabi) published the paper for K2-Horizon (arXiv:2611.21003), a fleet of six fully open language models, from a 0.9B dense checkpoint up to a 375B-A23B mixture-of-experts flagship built for coding and agentic work. Details at ifm.ai/k2.

Every fact below comes from the K2-Horizon paper (arXiv:2611.21003, submitted November 16, 2026) and IFM's page at ifm.ai/k2. I haven't run these models hands-on — this is researched analysis of the release, not a benchmark review. Where a claim is the paper's own, I say so.

The fleet, in plain terms

Six models, one shared design language, sized for different jobs:

  • 0.9B dense — the small one, for on-device and edge experiments
  • 3.7B and 7.0B dense — the practical middle, cheap to run and fine-tune
  • 32B dense — the serious single-GPU-class workhorse
  • 36B-A4B MoE — only 4B active parameters, and the paper says it nearly matches the 32B dense model trained on the same data
  • 375B-A23B MoE — the flagship, 23B active parameters, aimed at coding and agentic work

Four of the six share one 22-trillion-token pretraining corpus and recipe, which is the kind of controlled setup researchers dream about: same data, same recipe, different scales and sparsity. You can actually study what scale does, instead of guessing across releases that changed everything at once.

The models are built for general-purpose reasoning and agentic work — math and science, software engineering, deep research, office automation. The paper says they lead or are competitive with strong open models of their size on agentic and coding benchmarks. That's the paper's claim, not mine, and I won't dress it up with numbers I haven't verified.

Three technical ideas worth knowing

MoVA — Mixture-of-Value Attention. The paper applies mixture-of-experts thinking to the attention value projections. It's the kind of idea that sounds obvious once you read it and is genuinely new in practice. The 36B-A4B model uses it, and the headline result is that it nearly matches a 32B dense model on the same data with roughly one-seventh the active parameters. If that holds up in independent testing, the small-MoE shape gets a lot more interesting for anyone serving models on a budget.

Softplus attention gating. A more flexible, theoretically motivated gate on attention, also in the 36B-A4B. Honest admission: this is the part of the paper I can't fully evaluate without reading the math carefully, and most of us will feel the same. What matters practically is that it's part of why the small MoE punches above its active parameter count.

Uno — diffusion adapters for faster decoding. Uno is a lossless diffusion adapter that speeds up decoding by up to 2.2x, according to the paper, without needing a separate draft model. Speculative decoding usually means maintaining a second small model; Uno skips that. If you're serving an agent that makes dozens of sequential tool calls, decoding speed is the tax you pay on every step — a 2x speedup there is worth more than a few benchmark points.

The real news: they opened the kitchen, not just the dish

Here's what makes this release different from every other "open weights" drop this year. IFM says the release follows the LLM360 definition of open source for transparency and reproducibility, and the list is long:

  • Final weights for all six models
  • Intermediate checkpoints from every training stage — not just the final model, the whole development history
  • Pretraining data and the data-construction recipes
  • Training logs
  • All training, evaluation and infrastructure code, including the xLLM framework for dense and MoE training and the RL360 reinforcement-learning stack
  • The pipelines that built the data, documented in detail — the part of model development that open releases almost never disclose

That last one is the sleeper. Those pipelines produced roughly 10 trillion synthetic pretraining tokens. Synthetic data at that scale, with the pipeline open, means other labs can actually study how synthetic data behaves as a major pretraining ingredient instead of theorizing about it. IFM calls it a glass box, not a black box, and for once the phrase fits the release.

This is the difference between "you can download our model" and "you can see how we built it, reproduce it, and build on it." Most open releases stop at the first sentence. K2-Horizon tries for the second.

The honest limits

  • The flagship is not a laptop download. A 375B-A23B MoE is a datacenter-class model. The practical entry points for most developers are the 0.9B, 3.7B and 7.0B models, and the 36B-A4B if you have the hardware for it. Don't let the flagship number distract you from the models you'll actually run.
  • "Lead or competitive" is the paper's claim. The models themselves launched on September 3, 2026 — the November 16 paper is the formal write-up, not the launch — and third parties have already run them (the 375B scores 47 on the Artificial Analysis Intelligence Index). Treat the paper's own benchmark claims as provisional until you've checked them against independent runs on your own tasks.
  • Openness has a compliance footnote. IFM is Abu Dhabi-based. If your team mirrors full training corpora or deploys in a regulated environment, check your data residency and export rules before pulling the whole release, not after.
  • Reproducing a 22T-token run is not a weekend project. The openness is real, but the compute to retrain from scratch is not in reach for most teams. The value for most of us is inspection, fine-tuning, and studying the checkpoints — not re-running pretraining.

What I'd actually do this week

If you build with open models, the move is to start small and inspect, not to chase the flagship:

  1. Pull the 7.0B or the 36B-A4B first. The 7B is the cheap experiment; the 36B-A4B is the interesting one if you want to see whether MoVA delivers in practice.
  2. Read the data pipeline docs before you fine-tune. The pipelines are the part nobody else publishes. Even if you never retrain, knowing how the synthetic data was built tells you what the model is likely good and bad at.
  3. Try Uno on an agentic workload. If you have an agent loop that makes many sequential calls, a decoding speedup without a draft model is a real serving win — test it on your own latency, not the paper's number.
  4. Compare the 36B-A4B against the 32B dense on your own tasks. Same data, same recipe, one-seventh the active parameters — that's a clean experiment, and it's the claim most worth checking yourself.

Sources: K2 Team, "K2-Horizon: A Frontier Fleet of Open Source Language Models," arXiv:2611.21003, submitted November 16, 2026; IFM, "K2 Horizon: Open-Source AI Models for Every Scale," ifm.ai/k2. Model sizes, architecture details, token counts and openness claims are from the paper and IFM's page; benchmark claims are the paper's own and have not been independently verified. Full version with FAQ on ToolsDig.

A 375B open model with the training data attached — is this the new bar for "open," or just the most transparent release of a very open year? And which size would you actually run?

Top comments (0)