Granite 4.1 LLMs: How They’re Built

An in-depth technical walkthrough of data engineering, pre-training, supervised fine-tuning, and reinforcement learning behind the Granite 4.1 LLMs.

Authors: Granite Team, IBM

TL;DR — Granite 4.1 is a family of dense, decoder‑only LLMs (3B, 8B, and 30B) trained on ~15T tokens using a multi‑stage pre‑training pipeline, including long‑context extension of up to 512K tokens. The models are further refined with supervised fine‑tuning on ~4.1M high‑quality curated samples and reinforcement learning via on‑policy GRPO with DAPO loss (Yu et al., 2025). Notably, the 8B instruct model matches or surpasses the previous Granite 4.0‑H‑Small (32B‑A9B MoE) despite using a simpler dense architecture with fewer parameters. All Granite 4.1 models are released under the Apache 2.0 license.

Building high‑quality small language models goes beyond simply scaling compute—it requires rigorous data curation throughout training. For Granite 4.1, we prioritized data quality over quantity, progressively refining the data mixture across five pre‑training stages. We further curated supervised fine‑tuning data using an LLM‑as‑Judge framework and applied a multi‑stage reinforcement learning pipeline to systematically strengthen performance in math, coding, instruction following, and general chat.

Granite 4.1 models use a decoder-only dense transformer architecture. The core design choices include Grouped Query Attention (GQA), Rotary Position Embeddings (RoPE), SwiGLU activations, RMSNorm, and shared input/output embeddings.

| Component | 3B Dense | 8B Dense | 30B Dense | |---|---|---|---| | Embedding size | 2560 | 4096 | 4096 | | Number of layers | 40 | 40 | 64 | | Attention head size | 64 | 128 | 128 | | Number of attention heads | 40 | 32 | 32 | | Number of KV heads | 8 | 8 | 8 | | MLP hidden size | 8192 | 12800 | 32768 | | MLP activation | SwiGLU | SwiGLU | SwiGLU | | Position embedding | RoPE | RoPE | RoPE |

All three model sizes share the same training pipeline and data strategy, differing only in architecture dimensions.

Granite 4.1 is trained from scratch on approximately 15 trillion tokens using a five‑phase training strategy. Phases 1–2 focus on foundational pre‑training, phases 3–4 perform mid‑training with progressively higher‑quality data annealing, and phase 5 introduces long‑context training, extending the context window to 512K tokens. Each phase employs a distinct data mixture and learning‑rate schedule, gradually shifting from broad web‑scale data to more curated, domain‑specific content.

The first phase establishes broad language understanding using a general mixture of training data with a power learning rate schedule and warmup. Data composition: CommonCrawl ~59%, Code ~20%, Math ~7%, Technical ~10.5%, Multilingual ~2%, Domain Specific ~1.5%.

Phase 2 sharply increases the proportion of code and mathematical data, pivoting toward stronger reasoning capabilities while still maintaining general language coverage. Data composition: Math ~35%, Code ~30%, CommonCrawl-HQ ~12%, Synthetic ~9%, Technical ~10%, Multilingual ~3%, Domain ~1%.

Phase 3 transitions into mid-training with a more balanced, high-quality mixture and an exponential decay learning rate schedule. This is where we start blending in chain-of-thought and synthetic instruction data.

Phase 4 continues mid-training with a linear learning rate decay to zero, focusing the model on the highest-quality data available.

Phase 5 extends the context window from 4K to 512K through a staged long-context extension process (32K, 128K, 512K). To ensure the model natively handles long sequences without degrading short-context performance, we do a model merge after each LCE stage.

Supervised fine‑tuning (SFT) is what turns the base model into a reliable instruction‑following assistant. To address this, we apply a rigorous LLM‑as‑Judge framework alongside rule‑based filtering to curate high-quality samples. Together, the pipeline automatically assess each sample against structural, semantic, and behavioral criteria, fixing issues when possible and filtering out samples that fail to meet our quality standards.