Local AI
Local AI
Foundations
Local AI Hardware Check
Estimate what your machine can run before downloading models
48 of 66
What matters
Three numbers decide your local AI experience:
- VRAM on your GPU: the fast memory models live in.
- System RAM: used when a model does not fit in VRAM.
- Storage: model weights are 4 GB to 30 GB each.
Quick checks
macOS
system_profiler SPDisplaysDataType | grep "VRAM"
Apple Silicon uses unified memory. Check total RAM:
sysctl hw.memsize
Windows
Open Task Manager → Performance → GPU. Look at "Dedicated GPU memory".
Linux
nvidia-smi free -h
Rule of thumb
| Model size | Quantization | VRAM needed | Good for |
|---|---|---|---|
| 7B | Q4 | 4–6 GB | Fast chat, simple coding |
| 13B | Q4 | 8–10 GB | Better reasoning |
| 70B | Q4 | 40+ GB | Near-frontier quality |
If you do not have a GPU
Modern CPUs can run small quantized models. It will be slower, but it works. Use Ollama or LM Studio and set GPU offload to 0 layers.
Storage tip
Keep models on an SSD. Loading a 10 GB weight file from a spinning disk is painful.