# Chat completions

POST /v1/chat/completions on mlx-lm's server takes a chat and returns the model's answer. Every field mlx-lm documents, with its default.

```endpoint
POST http://127.0.0.1:8080/v1/chat/completions
```

The fields below are the ones listed in mlx-lm's server guide ([SERVER.md](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md), mlx-lm 0.32.2). The API is meant to be similar to OpenAI's chat API; a field that isn't listed here isn't documented by mlx-lm.

## Request

| Parameter | Type | Notes |
| --- | --- | --- |
| `messages` | array | The conversation so far. Each message has a `role` (for example `user`, `assistant`) and `content` (the text). |
| `model` | string | Optional. A local path (relative to the folder the server was started in) or a Hugging Face repo id. |
| `role_mapping` | object | Optional. Custom role prefixes for the prompt; the defaults are used without it. |
| `stop` | string or array | Optional. Sequences on which generation stops. |
| `max_tokens` | integer | Optional. Most tokens to generate. Default `512`. |
| `stream` | boolean | Optional. Send the answer as it is generated. Default `false`. |
| `temperature` | number | Optional. Sampling temperature. Default `0.0`. |
| `top_p` | number | Optional. Nucleus sampling. Default `1.0`. |
| `top_k` | integer | Optional. Top-k sampling. Default `0` (off). |
| `min_p` | number | Optional. Min-p sampling. Default `0.0` (off). |
| `repetition_penalty` | number | Optional. Multiplicative penalty on repeated tokens. Default `0.0` (off). |
| `repetition_context_size` | integer | Optional. Window for the repetition penalty. Default `20`. |
| `presence_penalty` | number | Optional. Additive penalty on tokens that appeared before. Default `0.0` (off). |
| `presence_context_size` | integer | Optional. Window for the presence penalty. Default `20`. |
| `frequency_penalty` | number | Optional. Additive penalty in proportion to how often a token appeared. Default `0.0` (off). |
| `frequency_context_size` | integer | Optional. Window for the frequency penalty. Default `20`. |
| `logit_bias` | object | Optional. Token id → bias. |
| `logprobs` | integer | Optional. Return this many top tokens with their log probabilities for each output token, 1 to 10. |
| `adapters` | string | Optional. Path to low-rank adapters, relative to the folder the server was started in. |
| `draft_model` | string | Optional. A smaller model for speculative decoding; `null` unloads it. |
| `num_draft_tokens` | integer | Optional. Tokens the draft model predicts at once. Default `3`. |

## Response

| Field | Notes |
| --- | --- |
| `id` | A unique id for the chat. |
| `system_fingerprint` | A unique id for the system. |
| `object` | `chat.completion`, or `chat.completion.chunk` when streaming. |
| `model` | The model repo or path. |
| `created` | When the request was processed. |
| `choices` | A list of outputs, each with `index`, `logprobs` (`token_logprobs`, `tokens`, `top_logprobs`), `finish_reason` (`stop` or `length`) and `message`. |
| `usage` | `prompt_tokens`, `completion_tokens` and `total_tokens`. |

## Example

```bash tab=curl
curl localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }'
```

A `finish_reason` of `length` means the answer hit `max_tokens`: raise it and ask again.