Mellum2 Technical Report

JetBrains released Mellum 2, a 12B-parameter open-weight MoE model built for software engineering tasks.

It targets code generation, editing, debugging, reasoning, tool use, and conversational coding assistance.

The model uses efficiency-focused techniques like grouped-query attention, sliding window attention, and multi-token prediction, with only 2.5B active parameters per token.

Trained on about 10.6T tokens and extended to a 128K context window, it comes in base, instruct, and thinking checkpoints under Apache 2.0.