Skip to content
localmodel.run

text model · SmolLM2 · Windows

Can I run SmolLM2 360M on Nvidia GeForce RTX 4080 (16GB)?

Compatibility verdict VRAM threshold engine
Yes, it runsGPU accelerated

Yes. SmolLM2 360M runs on Nvidia GeForce RTX 4080 (16GB) at Q4_K_M (~1.2 GB of ~15 GB usable).

Needs ~1.2 GB Device usable ~15 GB

Runs at Q4_K_M using ~1.2 GB of ~15 GB usable. You have room for FP16 for higher quality.

Q4_K_M needed
~1.2 GB
Usable on device
~15 GB
Device memory
16 GB
Best quant
Q4_K_M

Run it

Install commands Windows

Pick your tool. All three load the same Q4_K_M weights.

Ollama
$ ollama run smollm2:360m
llama.cpp
$ llama-cli -hf bartowski/SmolLM2-360M-Instruct-GGUF:Q4_K_M
LM Studio
$ lms get bartowski/SmolLM2-360M-Instruct-GGUF

AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.

Model SmolLM2
Parameters
0.362B
Q4_K_M size
0.271 GB
Q8_0 size
0.386 GB
Context
2k
Ollama tag
smollm2:360m
Full SmolLM2 360M requirements →
Device Windows
Memory
16 GB vram
Usable for weights
~15 GB
Best runtime
vLLM (Linux) / Ollama (CUDA)
Best models for Nvidia GeForce RTX 4080 (16GB) →

You could also run

Run SmolLM2 360M on other hardware

Apple M1 (8GB)8GB RAM Laptop (CPU/iGPU only)iPhone 15 ProiPhone 16iPhone 16 ProGeneric Android Phone (8GB RAM)

FAQ

Can Nvidia GeForce RTX 4080 (16GB) run SmolLM2 360M?

Yes. SmolLM2 360M runs on Nvidia GeForce RTX 4080 (16GB) at Q4_K_M (~1.2 GB of ~15 GB usable).

How much memory does SmolLM2 360M need?

Nvidia GeForce RTX 4080 (16GB) has room to spare. At Q4_K_M the weights are ~0.271 GB; with KV cache and runtime overhead, budget ~1.2 GB at a 4k context.

What is the best tool to run SmolLM2 360M on Windows?

LM Studio for a simple setup; Ollama (CUDA) for the most speed. AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.

Sources

Memory figures are estimates. See methodology.