Mistral AI Releases Mistral Small 4: A 119B-Parameter MoE Model that Unifies Instruct, Reasoning, and Multimodal Workloads

Mistral Small 4 was released, mistral AI released Mistral Small 4 as part of the Mistral Small family.

Model architecture uses a Mixture-of-Experts design with 128 experts and 4 active experts per token model is a Mixture-of-Experts with 128 experts and 4 active experts per token.

Model size has 119B total parameters and 6B active parameters per token (8B including embedding and output layers) 119 billion total parameters; 6B active parameters per token or 8B including embeddings and outputs.

Context window supports a 256k context window the model supports a 256k token context window.

Input/output formats accepts text and image inputs and produces text output model accepts text and image inputs and returns text outputs.