MLX Studio: one desktop app to download, run, and serve MLX models on Apple Silicon

MLX Studio is a macOS desktop app that downloads models from the mlx-community catalog on Hugging Face, runs them through mlx-lm (text) and mlx-vlm (vision), and serves every installed model over an OpenAI-compatible API on 127.0.0.1. We built it as the model runtime for a system we are developing: any component that speaks the OpenAI protocol connects to it by changing one base URL, with no adapter code.

The target hardware is Apple Silicon, and the choice of MLX over llama.cpp or an Ollama install follows from it. MLX is Apple’s array framework written against Metal and unified memory, so weights load once into the same memory the GPU computes on, with no copy between CPU and GPU pools. On the 48 GB MacBook used for the screenshots below, the 4-bit build of Qwen2.5 7B occupies 4.0 GB of weights and streams replies at 35 tok/s.

The dashboard: installed and running models, a live unified-memory gauge, and the activity log.

From catalog to first reply

The whole workflow is four screens.

1

Catalog: search mlx-community, filter by quantization or vision support. Each card shows download size, estimated RAM, and a fit badge (Fits / Tight / Too big) computed against the memory free on this machine right now, so an impossible model is visible before the download starts.

2

Download: progress streams over SSE with byte counts and speed. Downloads pause, resume, and cancel at file boundaries, so a 14 GB model interrupted at 90% does not restart from zero. MLX Studio pulls up to four files in parallel, and the Hub throttles anonymous requests well below what a fast line can carry. A free read token from huggingface.co/settings/tokens, saved under Settings ▸ Hugging Face, raises those rate limits and keeps parallel downloads at full speed; it covers catalog searches too.

3

Start: the context length is a slider, and the memory estimate updates live as it moves: weights, KV cache (the part that grows with context), and runtime overhead, against the memory free at that moment. Qwen2.5 7B at 8,192 tokens comes to 5.0 GB. The Start button disables itself when the total cannot fit.

4

Chat: streaming replies with Markdown and code highlighting, persisted conversation history, and the measured tokens-per-second under each reply. The selector in the header switches between running models.

The OpenAI-compatible API

Every installed model is served on http://127.0.0.1:11535/v1 behind the OpenAI protocol, and the chat tab is one client of that API. A request for a model that is installed but not running loads it on the first call, so client code never has to check state. When a model starts, the app opens a Connect dialog with the base URL, the API key, and ready-to-paste snippets for the Python OpenAI SDK, LangChain, Java with LangChain4j (plain builder or Spring Boot starter properties), Rust with Rig, and curl.

The Connect dialog opens as soon as a model is up; snippets cover Python, LangChain, Java (LangChain4j), Rust (Rig), and curl.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:11535/v1", api_key="<MLX Studio API key>")
resp = client.chat.completions.create(
    model="qwen2.5-7b-instruct-4bit",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

The same endpoint works for Continue, Cursor, or any tool with a configurable OpenAI base URL. Our own system consumes it the same way. The app owns download, memory, and process lifecycle; consumers see a standard OpenAI server that runs offline.

Vision models

Repos with a vision tower (Qwen-VL, LLaVA, Gemma vision variants) load through mlx-vlm instead of mlx-lm; the app decides per model by reading its config.json. When the running model supports vision, the chat input shows an attach button and accepts pasted images, and the /v1 endpoint takes standard OpenAI image_url content parts, as base64 data URLs or http URLs. Qwen2.5-VL 3B (4-bit, 2 GB) described a test photograph correctly at 81 tok/s on the same MacBook.

Qwen2.5-VL 3B reads an attached photo and streams the description at 81 tok/s.

mlx-vlm 0.6.4 truncates every generation after the first one in a process to a single token. The first request works, so unit tests and one-shot CLI checks pass; the bug appeared only when we sent three consecutive requests through the running server. MLX Studio pins mlx-vlm>=0.6.3,<0.6.4, and the pin stays until a later release passes the same multi-request test.

Architecture

The Rust core spawns the Python sidecar on launch and injects a per-launch bearer token, so the local API is not an open port for any process on the machine. The sidecar can never outlive the app: on quit the core kills the child and sweeps the port, and the sidecar itself watches the app’s PID and exits if the app dies without cleanup. A force-quit leaves no orphan process holding 4 GB of weights.

Status

The code is on GitHub, MIT licensed, at version 0.4.0. CI builds an Apple Silicon DMG on every push; MLX runs on Metal and unified memory, so Apple Silicon is a hard requirement and there is no Intel build. The DMG has no prerequisites: the Python sidecar ships inside the bundle as a PyInstaller binary and the shell uses the WebView already in macOS, so on a Mac with an M-series chip the app runs as downloaded, no Python, Homebrew, or runtime install first. The next planned changes are versioned releases wired to the in-app updater, and multi-model memory budgeting so two models can run inside the same gauge.