# Models

GET /v1/models lists the models mlx-lm's server has locally. Which ones fit your Mac, and how to save memory on long chats.

## List models

```endpoint
GET http://127.0.0.1:8080/v1/models
```

```bash title="Request"
curl localhost:8080/v1/models -H "Content-Type: application/json"
```

The answer is a list of the models available locally ([SERVER.md](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md), mlx-lm 0.32.2). Each has:

| Field | Notes |
| --- | --- |
| `id` | The Hugging Face repo id. Send it as `model` to use that model. |
| `created` | When the model was created. |

## Choosing a model

The server takes a local path or a Hugging Face repo (`--model`, or `model` in a request). Builds made for MLX, often 4-bit, are published on Hugging Face; the guide's example is `mlx-community/Mistral-7B-Instruct-v0.3-4bit`.

### Will it fit? {#will-it-fit}

A model needs about its parameter count times the bits stored per weight, plus room for the context, and macOS lets the GPU use about three quarters of unified memory by default. [mlx.app/compare](https://mlx.app/compare) does that arithmetic for every Apple chip and popular open model, and [mlx.app/run](https://mlx.app/run) has a page per model with the smallest Mac that holds it. Both are estimates from published specifications.

## Long chats: a quantized KV cache

On long contexts the KV cache can take a lot of memory. `--kv-bits` stores it quantized:

```bash
mlx_lm.server --model <path_to_model_or_hf_repo> --kv-bits 4 --prefill-step-size 512
```

- Attention on a quantized cache is not fused, so it keeps a score matrix of `prefill_step_size × context_length`. Lower `--prefill-step-size` if that matrix would cost more than the cache saves.
- `--kv-group-size` sets the group size and `--quantized-kv-start` the token position where quantization starts (default `5000`).
- A quantized cache doesn't batch: with `--kv-bits` set, the server handles one request at a time.