ai/smollm2
Tiny LLM built for speed, edge devices, and local development
100K+
ai/smollm2 repository overview
SmolLM2
SmolLM2-360M is a compact language model with 360 million parameters, designed to run efficiently on-device while performing a wide range of language tasks. Trained on 4 trillion tokens from a diverse mix of datasets—including FineWeb-Edu, DCLM, The Stack, and newly curated filtered sources—it delivers strong performance in instruction following, knowledge, and reasoning. The instruct version was developed through supervised fine-tuning (SFT) on a blend of public and proprietary datasets, followed by Direct Preference Optimization (DPO) using UltraFeedback.
Intended uses
SmolLM2 is designed for:
- Chat assistants
- Text-extraction
- Rewriting and summarization
Characteristics
| Attribute | Details |
|---|---|
| Provider | Hugging Face |
| Architecture | Llama2 |
| Cutoff date | June 2024 |
| Languages | English |
| Tool calling | ✅ |
| Input modalities | Text |
| Output modalities | Text |
| License | Apache 2.0 |
Available model variants
| Model variant | Parameters | Quantization | Context window | VRAM¹ | Size |
|---|---|---|---|---|---|
ai/smollm2:latestai/smollm2:360M-Q4_K_M | 360M | IQ2_XXS/Q4_K_M | 8K tokens | 0.63 GiB | 256.35 MB |
ai/smollm2:135M-Q4_0 | 135M | Q4_0 | 8K tokens | 0.35 GiB | 85.77 MB |
ai/smollm2:135M-Q4_K_M | 135M | IQ2_XXS/Q4_K_M | 8K tokens | 0.36 GiB | 98.87 MB |
ai/smollm2:135M-F16 | 135M | F16 | 8K tokens | 0.51 GiB | 256.63 MB |
ai/smollm2:135M-Q2_K | 135M | Q2_K | 8K tokens | 0.34 GiB | 82.41 MB |
ai/smollm2:360M-Q4_0 | 360M | Q4_0 | 8K tokens | 0.59 GiB | 216.80 MB |
ai/smollm2:360M-Q4_K_M | 360M | IQ2_XXS/Q4_K_M | 8K tokens | 0.63 GiB | 256.35 MB |
ai/smollm2:360M-F16 | 360M | F16 | 8K tokens | 1.06 GiB | 690.24 MB |
¹: VRAM estimated based on model characteristics.
latest→360M-Q4_K_M
Use this AI model with Docker Model Runner
First, pull the model:
docker model pull ai/smollm2
Then run the model:
docker model run ai/smollm2
For more information on Docker Model Runner, explore the documentation.
Benchmark performance
| Category | Benchmark | Score |
|---|---|---|
| Reasoning | HellaSwag | 54.5 |
| Science | OpenBookQA | 37.4 |
| ARC | 53.0 | |
| Reasoning | PIQA | 71.7 |
| CommonsenseQA | 38.0 | |
| Winogrande | 52.5 | |
| Popular Aggregated Benchmark | MMLU (cloze) | 35.8 |
| TriviaQA (held-out) | 16.9 | |
| Math | GSM8K (5-shot) | 3.2 |
Links
Tag summary
Content type
Model
Digest
sha256:d2df8c834…
Size
100.6 MB
Last updated
6 months ago
docker model pull ai/smollm2:135M-Q4_K_MThis week's pulls
Pulls:
9,844
Last week