MLX Studio Local models on this Mac

Run MLX models on your Mac and call them over an OpenAI-compatible API.

MLX Studio is a macOS app for Apple Silicon. It downloads models from the mlx-community organisation on Hugging Face, checks each one against the Mac's unified memory before it loads, and runs it with mlx-lm, or with mlx-vlm for vision models. You chat with the models in the app, and other programs call the same models at 127.0.0.1:11535/v1.

13 GB free for models

of 48 GB

  • Qwen3 8B 7.2 GB 15%
  • Qwen2.5 VL 3B Instruct 3.6 GB 7%
  • macOS and apps 17 GB 36%
  • Safety reserve 7.2 GB 15%
  • Free 13 GB 26%
Engine
mlx-lm for text, mlx-vlm for vision
API
OpenAI-compatible, /v1 on port 11535
App
Tauri 2 shell, Vue 3 frontend, Python FastAPI sidecar
Requires
macOS 12 or later on Apple Silicon

Overview ⌘1

Overview shows where the Mac's memory is and which models are loaded.

The dial draws the Mac's unified memory as a 270° ring. Each loaded model is an arc in its own colour. macOS and other apps are a dim arc, and the 15% safety reserve is hatched. The empty track that remains is the memory a new model can take, and the figure in the centre is its size. Point at an arc or at its legend row to read that segment's name and size.

Three tiles under the legend show CPU load over the last minute, free disk space in the models folder, and swap. When swap passes 1 GB, the tile turns amber and a warning suggests stopping a model or restarting it with a shorter context, because generation slows down once macOS pages memory to disk. Each running model has Chat, Use from code and Stop, and a bar with its share of memory in the colour of its arc. The activity column lists downloads, starts and stops.

Data
GET /system/stats/stream, one reading per second over SSE
Free for models
Available memory minus 15% of total
Model colours
Four tones. A model takes the lowest tone no loaded model uses
Swap warning
Above 1 GB

Apple Silicon · 48 GB unified memory

Overview

Browse catalog New chat
13 GB free for models

of 48 GB

Where the memory is

  • Qwen3 8B 7.2 GB 15%
  • Qwen2.5 VL 3B Instruct 3.6 GB 7%
  • macOS and apps 17 GB 36%
  • Safety reserve 7.2 GB 15%
  • Free 13 GB 26%
CPU
57%
Disk free
312 GB
Swap
0 B

Running 2

Qwen3 8B

8B 4bit 16K context
Chat Use from code Stop
7.2 GB · 15% of memory

Qwen2.5 VL 3B Instruct

3B 4bit Vision 8K context
Chat Use from code Stop
3.6 GB · 7% of memory

Activity

  1. Started Qwen2.5 VL 3B Instruct

    4 min ago
  2. Downloaded mlx-community/Meta-Llama-3.1-8B-Instruct-4bit

    12 min ago
  3. Started Qwen3 8B

    1 h ago
  4. Stopped Llama 3.1 8B Instruct

    1 h ago
  5. Started download: mlx-community/gemma-3-27b-it-4bit

    2 h ago

Mocked data Point at an arc or a legend row. The light moving along the Qwen3 8B arc means that model is generating.

ModelCore, in the rail, the lists, the palette and each reply

One instrument draws a model's state everywhere the model appears.

The core is a ring around a disc. It appears in the rail, in the model lists, in the command palette, in the chat model picker and next to every reply. The six states change how much of the ring is lit, how full the disc is, and whether anything moves. Only loading and generating animate continuously. A running model is drawn still, because a loaded model that is not generating does no work.

A transition to running plays one arrival, in which the disc overshoots and a ripple leaves the ring. A transition to failed shakes the core once. Neither plays when a view first opens, only on a change of state. The buttons below drive one core through a load.

Colour
The model's memory tone, the same as its arc on the dial
Source
The model's status, a start or stop request in flight, and whether Chat is streaming from it
Detail
The twelve bezel ticks are drawn from 28 px up
  • Not loaded

    A hairline ring and a small dot. The model is installed and uses no memory.

  • Loading

    A comet orbits the ring and lights the ticks while the weights stream into memory.

  • Running

    The ring closes in the model's memory colour and the core lights. Nothing moves.

  • Generating

    The core pulses and a bright arc sweeps the ring while tokens stream.

  • Stopping

    The ring drains while the unload request is in flight.

  • Failed

    The ring breaks open in red. The model log has the cause.

Not loaded

Catalog ⌘4

The catalog searches mlx-community and sizes each result against this Mac.

The search runs against the Hugging Face Hub 300 ms after you stop typing. Filters limit the results to 4-bit or 8-bit weights, vision models, instruct models, or models that fit in free memory now. Results sort by downloads in the last 30 days, by likes or by last update, 30 at a time. ⌘F focuses the search, and the command palette can send what you typed there.

Each row has a fit gauge. The track is what the Mac can give a model at all, which is total memory minus the reserve. The bar is what the model needs, and the white notch marks what is free right now. A model that is not downloaded has no config.json to read yet, so its estimate is the weight size from the parameter count and the quantization, plus 10% for the KV cache and 15% for runtime overhead.

Fits
Needs no more than is free now
Tight
Needs more than is free now, and less than total minus the reserve. It starts, and macOS reclaims cached memory to make room
Too big
Needs more than the Mac can give a model. Start is disabled
Row button
Download, a progress fill while it runs, Paused, Retry or Installed

MLX conversions from Hugging Face, sized for this Mac

Catalog

4-bit 8-bit Vision Instruct Fits this Mac Most downloaded

Qwen3 8B

8B 4bit 412.6K 58
4.7 GB Fits
Installed

Gemma 3 12B it

12B 4bit Vision Instruct 188.2K 41
7.0 GB Fits
Download

Gemma 3 27B it

27B 4bit Vision Instruct 96.4K 37
16 GB Tight
62%

Mistral Small 3.1 24B Instruct 2503

24B 8bit Instruct 31.9K 22
28 GB Tight
Paused 31%

Llama 3.3 70B Instruct

70B 8bit Instruct 24.7K 19
81 GB Too big
Download

Mocked data On this 48 GB Mac, 13 GB is free for models and 41 GB is the most any model can get. Llama 3.3 70B Instruct at 8-bit needs 81 GB.

Start dialog, from Overview, Models, Chat or ⌘K

Starting a model shows what it costs in memory before it loads.

Start opens a dialog with a context-length slider in steps of 1,024 tokens, up to the model's max_position_embeddings. The estimate is recomputed 180 ms after the slider stops. For an installed model, the weights are the size of the safetensors files on disk, and the KV cache comes from the model's config.json.

The strip under the figure is the dial's ledger, one cell per GB. The model you are about to start is drawn as outlined cells in the colour it will get. Memory it needs beyond what the Mac has is drawn as red cells past the end of the strip. Start is disabled only when the model is too big at the chosen context. The Reasoning switch sets enable_thinking, for models such as Qwen3 and DeepSeek R1 that reason before they answer.

KV cache
2 × layers × context × KV heads × head dim × 2 bytes
Overhead
15% of the weights
Presets
4K, 8K, 16K, 32K, 64K and 128K, up to the model's maximum
Endpoint
POST /models/{id}/start with context_length and enable_thinking

5 installed · 2 running · 17 GB on disk

Models

Get more models

Running 2

Qwen3 8B

mlx-community/Qwen3-8B-4bit 8B 4bit 4.3 GB
Chat Stop

Qwen2.5 VL 3B Instruct

mlx-community/Qwen2.5-VL-3B-Instruct-4bit 3B 4bit Vision 2.9 GB
Chat Stop

Installed 3

DeepSeek R1 Distill Qwen 7B

mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit 7B 4bit 4.0 GB
Start

Llama 3.1 8B Instruct

mlx-community/Meta-Llama-3.1-8B-Instruct-4bit 8B 4bit 4.5 GB
Start

Whisper Large v3 Turbo

Not a chat model
mlx-community/whisper-large-v3-turbo 800M 1.5 GB

Mocked data Llama 3.1 8B Instruct at a 16K context: 4.5 GB of weights and 2.0 GB of KV cache. The outlined cells take the third model colour, the first one no loaded model uses.

Downloads ⌘5

Downloads fetch only the files a model needs, and resume where they stopped.

A download lists the repo's files on the Hub, keeps the ones needed to run the model, and fetches them one at a time. Progress and speed reach the app over SSE, and the speed is smoothed over samples of the size on disk. Pause and Cancel are checked between files, so a pause takes effect when the current file finishes. On resume or retry, files that are already complete on disk are skipped.

A second Download of the same repo returns the job that is already running, because two workers writing one folder would corrupt it. A failed job keeps its files and retries from its row. Cancel asks first and says how many bytes it deletes. Clear finished removes the rows of completed jobs and leaves the installed models alone. The badge on Downloads in the rail counts the jobs that are downloading or queued.

Stream
GET /downloads/stream
Controls
POST /downloads/{id}/pause, POST /downloads/{id}/resume, DELETE /downloads/{id}
Completion
The worker registers the model as installed, whether or not the app is listening
Updates
Check for updates writes into the installed folder. Cancelling it never deletes the weights already there

1 in progress · 48.3 MB/s total

Downloads

Clear finished

gemma 3 27b it 4bit

10 GB of 16 GB

mlx-community/gemma-3-27b-it-4bit

48.3 MB/s, about 2 min left

Qwen2.5 Coder 14B Instruct 4bit

7.8 GB

mlx-community/Qwen2.5-Coder-14B-Instruct-4bit

Waiting to start

Mistral Small 3.1 24B Instruct 2503 8bit

7.8 GB of 25 GB

mlx-community/Mistral-Small-3.1-24B-Instruct-2503-8bit

Paused

Resume

Phi 4 mini instruct 4bit

973 MB of 2.2 GB

mlx-community/Phi-4-mini-instruct-4bit

Connection reset while fetching model.safetensors

Retry

Meta Llama 3.1 8B Instruct 4bit

4.5 GB of 4.5 GB

mlx-community/Meta-Llama-3.1-8B-Instruct-4bit

Installed

Start

Mocked data One job in each state: downloading, queued, paused, failed and finished. The light on the bar moves only while bytes arrive.

Models ⌘3

Models lists what is installed on disk and what is loaded in memory.

Running models sit in a lit group above the installed ones. Start and Stop are always visible. Use from code, the model log, Check for updates and Delete appear on the row under the pointer, so a list of ten models stays a list of ten names. Delete asks first, names the repo and the bytes it frees, and unloads a running model before it removes the weights.

A repo with no chat template, such as Whisper or an embedding model, is marked Not a chat model. It has no Start button, and neither Chat nor the API can use it. The log dialog follows the model's log live: context length and estimated memory at load, load time, prompt speed and peak memory per generation.

A result that lands outside the view you are on, such as a model finishing its load, a delete or a failure, appears as a toast in the bottom-right corner. Errors stay on screen longer than successes.

5 installed · 2 running · 17 GB on disk

Models

Get more models

Running 2

Qwen3 8B

mlx-community/Qwen3-8B-4bit 8B 4bit 4.3 GB
Chat Stop

Qwen2.5 VL 3B Instruct

mlx-community/Qwen2.5-VL-3B-Instruct-4bit 3B 4bit Vision 2.9 GB
Chat Stop

Installed 3

DeepSeek R1 Distill Qwen 7B

mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit 7B 4bit 4.0 GB
Start

Llama 3.1 8B Instruct

mlx-community/Meta-Llama-3.1-8B-Instruct-4bit 8B 4bit 4.5 GB
Start

Whisper Large v3 Turbo

Not a chat model
mlx-community/whisper-large-v3-turbo 800M 1.5 GB

Qwen2.5 VL 3B Instruct is running

Loaded into memory and answering on the local API.

Open chat

Mocked data The pointer is on DeepSeek R1 Distill Qwen 7B, so its row shows the four tools. The toast reports a load started from another page.

Chat ⌘2

Chat uses the running models and keeps every conversation on the Mac.

The picker in the header lists the running models. A message goes to the same /v1/chat/completions endpoint external clients use, and the reply is rendered once per animation frame while it streams. A model that reasons first, such as Qwen3 or DeepSeek R1, shows a Thinking label while it reasons. The reasoning then folds into a Thought for N s control above the answer.

Code blocks are highlighted and each has a copy button. A finished reply shows copy, regenerate, tokens per second and time to first token. When a reply stops at the token limit, a notice under it says so and names the Longest reply setting. With a vision model selected, the composer accepts up to four images from the picker, a paste or a drop.

Conversations are grouped by date, filtered by title and renamed from the header. The system prompt, the temperature and the longest reply apply to every chat and are kept across launches.

Endpoint
POST /v1/chat/completions with stream: true
Storage
SQLite, through the sidecar's /conversations endpoints
Stop
Aborts the request. The server sees the disconnect and stops generating
Empty chat
Four prompt starters, or the installed models to start when none is running

Retry decorator with backoff

Qwen3 8B
Write a Python retry decorator with exponential backoff and full jitter. Keep it under 20 lines.
Thought for 4 s

The decorator retries the listed exceptions and doubles the ceiling of the wait on each attempt. Each sleep is a random value between zero and that ceiling, so many clients that fail together do not retry together.

python Copy code
import random, time
from functools import wraps

def retry(attempts=5, base=0.2, exceptions=(Exception,)):
    def decorate(fn):
        @wraps(fn)
        def wrapper(*args, **kwargs):
            for attempt in range(attempts):
                try:
                    return fn(*args, **kwargs)
                except exceptions:
                    if attempt == attempts - 1:
                        raise
                    time.sleep(random.uniform(0, base * 2 ** attempt))
        return wrapper
    return decorate

The last attempt re-raises the original exception, so the caller sees the real error.

31.8 tok/s · 0.4 s to first token
Add a max_delay cap.
Thinking
Message Qwen3 8B

send · new line

Mocked data The thread is scrolled to the latest message. The first reply is finished and its reasoning is folded above it. The second is still reasoning, so its core and the Qwen3 8B core in the rail are both generating.

Local API and Use from code

Every installed chat model is served on an OpenAI-compatible API.

The sidecar serves /v1/models and /v1/chat/completions on 127.0.0.1:11535. /v1/models lists every installed chat model. A request for an installed model that is not running loads it first and then answers, so a client never has to start a model through the app. Requests are checked against the API key shown in Settings. The app's own calls to the rest of the sidecar use a separate token that the Rust core generates at each launch.

Tool calling follows the OpenAI format: tools, tool_calls, messages with the tool role, streamed deltas and finish_reason: "tool_calls". It works with models whose chat template declares a tool format that mlx-lm can parse, such as Qwen, Mistral, GLM, Gemma and Kimi. A request with tools to any other model returns 400. Vision models accept image_url content parts, as base64 data URLs or http URLs.

Generation runs on a single MLX thread, so two requests that arrive together are answered one after the other. Use from code opens the dialog below with the base URL, the key, the model id and a working client in Python, LangChain, Java with LangChain4j, Rust with Rig, or curl. The dialog remembers the last language you chose.

Base URL
http://127.0.0.1:11535/v1
Auth
Authorization: Bearer mlx-studio-local
Endpoints
GET /v1/models, POST /v1/chat/completions
Not running
Loaded on the first request, then answered
Tools without a format
400

5 installed · 2 running · 17 GB on disk

Models

Get more models

Running 2

Qwen3 8B

mlx-community/Qwen3-8B-4bit 8B 4bit 4.3 GB
Chat Stop

Qwen2.5 VL 3B Instruct

mlx-community/Qwen2.5-VL-3B-Instruct-4bit 3B 4bit Vision 2.9 GB
Chat Stop

Installed 3

DeepSeek R1 Distill Qwen 7B

mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit 7B 4bit 4.0 GB
Start

Llama 3.1 8B Instruct

mlx-community/Meta-Llama-3.1-8B-Instruct-4bit 8B 4bit 4.5 GB
Start

Whisper Large v3 Turbo

Not a chat model
mlx-community/whisper-large-v3-turbo 800M 1.5 GB
zsh
$ curl -N http://127.0.0.1:11535/v1/chat/completions \
    -H "Authorization: Bearer mlx-studio-local" \
    -H "Content-Type: application/json" \
    -d '{"model": "llama-3.1-8b-instruct-4bit", "stream": true,
         "messages": [{"role": "user", "content": "Say hi"}]}'

# llama-3.1-8b-instruct-4bit was installed and not running. It loads first, then streams.
data: {"object": "chat.completion.chunk", "choices": [{"delta": {"role": "assistant"}}], …}
data: {"object": "chat.completion.chunk", "choices": [{"delta": {"content": "Hi"}}], …}
data: {"object": "chat.completion.chunk", "choices": [{"delta": {"content": "!"}}], …}
data: {"object": "chat.completion.chunk", "choices": [{"delta": {}, "finish_reason": "stop"}], …}
data: [DONE]

Command palette ⌘K

⌘K reaches every page, every model and every app action.

The palette lists the six pages, the commands for each model and the app actions. A running model offers Chat and Stop. An idle model offers Start, which opens the Start dialog. Matching is fuzzy and ranks matches at the start of a word first. Whatever you type can also be sent to the catalog search. The arrow keys move, Enter runs the selected command, and Escape closes the palette.

K Open the command palette
1 Overview. ⌘2 to ⌘5 open Chat, Models, Catalog and Downloads
, Settings
N New chat, unless a reply is still streaming
F Focus the catalog search
esc Close the dialog or the palette
Q Quit, which also stops the engine

1 in progress · 48.3 MB/s total

Downloads

Clear finished

gemma 3 27b it 4bit

10 GB of 16 GB

mlx-community/gemma-3-27b-it-4bit

48.3 MB/s, about 2 min left

Qwen2.5 Coder 14B Instruct 4bit

7.8 GB

mlx-community/Qwen2.5-Coder-14B-Instruct-4bit

Waiting to start

Mistral Small 3.1 24B Instruct 2503 8bit

7.8 GB of 25 GB

mlx-community/Mistral-Small-3.1-24B-Instruct-2503-8bit

Paused

Resume

Phi 4 mini instruct 4bit

973 MB of 2.2 GB

mlx-community/Phi-4-mini-instruct-4bit

Connection reset while fetching model.safetensors

Retry

Meta Llama 3.1 8B Instruct 4bit

4.5 GB of 4.5 GB

mlx-community/Meta-Llama-3.1-8B-Instruct-4bit

Installed

Start

Mocked data “qw” matches both running Qwen models. The last row sends the text to the catalog.

Settings ⌘,

Settings apply at once and are saved on the Mac.

Appearance follows the system or forces light or dark. The light theme keeps the layout on a paper-coloured canvas and uses the lime only as a fill, never as text. Chat holds the defaults described above. Local API shows the base URL and the key to copy. A Hugging Face token is optional. It raises the Hub's rate limits and gives access to gated repos. Storage shows the models folder, and Memory draws the fit rule that the catalog and the Start dialog apply.

Saved on this Mac and applied at once

Settings

Appearance

System follows your Mac's light or dark setting.

System Light Dark

Chat

Applied to every chat in the app. Apps using the API send their own.

System prompt
Answer briefly. Put code in fenced blocks with the language named.
Temperature0.70
FocusedVaried
Longest reply4,096 tokens

Reasoning models spend part of this on thinking.

Local API

OpenAI-compatible. The address and key stay the same across launches.

Base URL
http://127.0.0.1:11535/v1
API key
mlx-studio-local

Use a model's id, such as qwen3-8b-4bit, as the model name. Stopped models load on the first request.

Hugging Face

Optional. A token raises rate limits and unlocks gated models.

A token is saved. Paste a new one to replace it.
Save Remove

Create a read token at huggingface.co/settings/tokens.

Storage

Where model weights are kept.

~/Library/Application Support/MLX Studio/models

Memory

How MLX Studio decides whether a model fits.

Fits — needs less than is free Tight — macOS reclaims cache Reserve, 15%

15% of memory is kept for macOS and your other apps. A model “fits” when it needs less than what is free after that reserve; “tight” models still start, because macOS reclaims cached memory, but other apps may slow down.

Mocked data Each theme choice is drawn as a small window in that theme.

Inside the bundle

MLX Studio runs as three processes inside one app bundle.

MLX, mlx-lm and huggingface_hub are Python libraries, so the engine runs in a Python sidecar. The shell is Tauri, which uses the system WebView instead of bundling Chromium. In a packaged build the sidecar is a PyInstaller binary inside the bundle, so nothing has to be installed first: no Python and no Homebrew. In development the three run as separate processes, and the frontend falls back to localhost defaults when it runs in a plain browser.

  1. Tauri core

    Rust

    Owns
    The window, the menu-bar item and the sidecar's lifecycle.
    Talks to
    Starts the sidecar with a port and two tokens, and hands them to the WebView through get_runtime_config. Stops the sidecar on quit.
  2. Frontend

    Vue 3, TypeScript, Vite

    Owns
    Every screen, the instruments and the keyboard map.
    Talks to
    Calls the sidecar over HTTP and SSE on localhost. It never reads model files and never calls Hugging Face.
  3. Sidecar

    Python, FastAPI, mlx-lm, mlx-vlm

    Owns
    Catalog search, downloads, the engine, SQLite and the /v1 API.
    Talks to
    Calls the Hugging Face Hub, writes to the models folder and computes on the GPU through MLX and Metal.

Install and build

Install the disk image, or build the app from source.

MLX Studio.app

macOS 12 or later on Apple Silicon. Rebuilt from main after CI passes.

Release page Download the disk image

The build is not signed or notarized, so macOS reports the downloaded app as damaged. After copying it to Applications, clear the quarantine flag once.

xattr -dr com.apple.quarantine "/Applications/MLX Studio.app"

Develop

Run the sidecar and the Tauri shell in two terminals. In dev, the Rust core passes the real port and tokens to the frontend.

# terminal 1
cd sidecar
uv sync --extra mlx
uv run uvicorn mlxstudio.main:app --port 11535 --reload

# terminal 2
pnpm install
pnpm tauri:dev

Build a release

The sidecar is bundled first, then Tauri builds the app and the disk image. make dmg-native does both and installs the app into Applications on this Mac.

pnpm sidecar:build   # PyInstaller
pnpm tauri:build     # .app and .dmg

Continuous integration

Every push to main runs lint, typecheck and vitest on the frontend, and pytest and ruff on the sidecar. When all pass, a macos-14 runner builds the app and replaces the release tagged latest. Pull requests run the same checks without releasing.

ToolVersionUsed for
macOS12+On Apple Silicon. MLX does not run on Intel Macs
Node18+Vite build of the Vue frontend, with pnpm
RuststableThe Tauri 2 shell, plus the Xcode command-line tools
uvanyThe Python sidecar and its lockfile

On a Mac without Apple Silicon the sidecar runs with a stub engine, so the interface can still be developed there.