Getting Started

node-mlx is the fastest way to run large language models in Node.js on Apple Silicon.

Requirements: macOS 14+ (Sonoma) on Apple Silicon (M1/M2/M3/M4), Node.js 20+

Installation

npm install node-mlx

Or with other package managers:

pnpm add node-mlx
yarn add node-mlx

Quick Start

Try the CLI

The fastest way to test node-mlx:

npx node-mlx "What is 2+2?"

For interactive chat mode:

npx node-mlx --model qwen --interactive

Basic API Usage

import { generate } from "node-mlx"

// One-shot generation (loads, generates, unloads automatically)
const result = generate("qwen", "Explain quantum computing in simple terms:", {
  maxTokens: 200,
  temperature: 0.7
})

console.log(result.text)
console.log(`${result.tokensPerSecond} tokens/sec`)

Keeping the Model Loaded

For multiple generations, keep the model loaded:

import { loadModel } from "node-mlx"

// Load a model (downloads automatically on first use)
const model = loadModel("phi")

// Generate multiple responses
const result1 = model.generate("What is 2+2?")
const result2 = model.generate("What is the capital of France?")

// Clean up when done
model.unload()

How Model Loading Works

First use: Model downloads from HuggingFace (~2-8 GB depending on model)
Cached: Models are stored in ~/.cache/huggingface/ for future use
GPU ready: Model loads directly into Apple Silicon unified memory

// First call - downloads and caches
const model = loadModel("mlx-community/phi-4-4bit")
// ⏳ Downloading... (one time only)

// Second call - instant from cache
const model2 = loadModel("mlx-community/phi-4-4bit")
// ⚡ Ready immediately

Next Steps

Model Guide - Choose the right model for your use case
API Reference - Full API documentation

Getting Started

On this page