Model Guide

node-mlx supports a wide range of models from leading AI providers. Use short aliases or full HuggingFace model IDs.

Recommended Models

Qwen 3 (Default)

Best for: General use, multilingual tasks, balanced quality/speed

loadModel("qwen3") // Qwen3-4B (default, best balance)
loadModel("qwen3-8b") // Larger, better quality
loadModel("qwen3-32b") // Premium quality

Variant	Parameters	Memory	Speed	Best For
`qwen3`	4B	~5 GB	120 tok/s	Recommended
`qwen3-8b`	8B	~8 GB	90 tok/s	Complex tasks
`qwen3-32b`	32B	~20 GB	40 tok/s	Premium quality

Phi 4 (Microsoft)

Best for: High quality reasoning, coding tasks

loadModel("phi") // Phi-4 (default, 14B, highest quality)
loadModel("phi4") // Phi-4 (alias)

Gemma 3 (Google)

Best for: Latest architecture, wide range of sizes

loadModel("gemma3") // Gemma-3-4B (default)
loadModel("gemma3-12b") // Gemma-3-12B
loadModel("gemma3-27b") // Gemma-3-27B (largest)

Gemma 3n (Google)

Best for: Efficient architecture, good quality/memory ratio

loadModel("gemma3n") // Gemma-3n-E4B (default)
loadModel("gemma3n-e2b") // Gemma-3n-E2B (smaller)

Llama 4 (Meta)

Best for: Advanced reasoning, multilingual, large context

Note: Requires HuggingFace authentication. Run huggingface-cli login first.

loadModel("llama4") // Llama-4-Scout (default)
loadModel("llama4-17b") // Llama-4-Scout-17B-16E

Ministral (Mistral AI)

Best for: Efficient like Gemma3n, good quality/size ratio

loadModel("ministral") // Ministral-8B (default)
loadModel("ministral-8b") // Ministral-8B
loadModel("ministral-14b") // Ministral-14B (better quality)

GPT-OSS (OpenAI)

Best for: MoE architecture, experimental

loadModel("gptoss") // GPT-OSS-20B (default)
loadModel("gptoss-20b") // GPT-OSS-20B MoE

SmolLM 3 (Hugging Face)

Best for: Tiny footprint, edge devices

loadModel("smollm3") // SmolLM3-3B (default)
loadModel("smollm3-3b") // SmolLM3-3B

Using HuggingFace Models

You can use any compatible model from mlx-community:

loadModel("mlx-community/Mistral-7B-Instruct-v0.3-4bit")
loadModel("mlx-community/gemma-3-4b-it-4bit")
loadModel("mlx-community/phi-4-4bit")

Model Quantization

Most models come in two variants:

Variant	Memory	Quality	Speed
bf16	~2× size	100% (reference)	Baseline
4-bit	~4× smaller	~97-99%	Often faster

When to use 4-bit:

Limited RAM (8-16 GB)
Larger models (7B+)
General conversation, creative writing

When to use bf16:

Math, logic, coding tasks
Maximum quality critical
Smaller models where memory isn't a concern

// 4-bit: ~4 GB RAM
loadModel("mlx-community/phi-4-4bit")

// bf16: ~28 GB RAM
loadModel("mlx-community/phi-4-bf16")

Supported Architectures

Architecture	Example Models	Status
Qwen3	Qwen3 4B–32B	✅ Full support
Llama	Llama 4, Mistral	✅ Full support
Phi3	Phi-4	✅ Full support
Gemma3	Gemma 3 (4B–27B)	✅ Full support
Gemma3n	Gemma 3n E2B/E4B	✅ Full support
Mistral3	Ministral 3 (3B–14B)	✅ Full support
SmolLM3	SmolLM3 3B	✅ Full support
GPT-OSS	GPT-OSS 20B/120B MoE	✅ Full support

Model Guide

On this page