MLXDocs
Local AI · mlx-lm

Local AI

Markdown

Run an open model on your own Mac with mlx-lm's server, an HTTP API meant to be similar to OpenAI's chat API, at http://127.0.0.1:8080.

mlx-lm is the language-model package of the MLX project, built on Apple's MLX framework for Apple silicon. It comes with mlx_lm.server, an HTTP server for one model on your own Mac. Everything in this section is taken from mlx-lm's own server guide (SERVER.md) and its --help, checked against mlx-lm 0.32.2 from PyPI on 1 November 2026. When mlx-lm and these pages disagree, mlx-lm is right.

You don't need MLX Node for this. MLX Node is how a Mac earns for the network while it's idle; running a model for yourself is separate.

How it fits together

  • One command, one model. mlx_lm.server --model <path_to_model_or_hf_repo> loads the model and serves it. A Hugging Face repo is downloaded the first time, unless it is already in the local cache.
  • On this Mac. By default it listens on 127.0.0.1, port 8080 (--host and --port change them; mlx_lm.server --help lists every option).
  • Not for production. mlx-lm says so itself: the server "only implements basic security checks". Keep it on 127.0.0.1 unless you put something in front of it.

Endpoints

Base URL http://127.0.0.1:8080.

MethodPathWhat it does
POST/v1/chat/completionsChat, with optional streaming. See Chat completions.
GET/v1/modelsThe models available locally. See Models.
Check it's running
curl localhost:8080/v1/models -H "Content-Type: application/json"
Quickstart
Install mlx-lm, start a model and call it.
Chat completions
Every request and response field mlx-lm documents.
Models
Listing models, picking one that fits your Mac, and saving memory on long chats.