How to Run Mistral Large 4: The Beginner's Guide (Weights, Hardware, and the First Local Run)

CodexFuse Team · 2026-11-03T12:38:20.855Z · 15 min read min read

The complete beginner arc for the open weights: the license catch, honest hardware math, the vLLM 8-GPU recipe, the llama.cpp GGUF route, and the API shortcut.

How to Run Mistral Large 4: The Beginner's Guide (Weights, Hardware, and the First Local Run)

Learning how to run Mistral Large 4 is unlike running any open-weight model you have touched before, because this one is enormous: roughly one trillion parameters, two official quantized checkpoints that weigh in at over a terabyte and 610 gigabytes respectively, and a license with a genuine catch. Mistral released the weights on October 29, 2026 under the codename "Le Chonk," and for the first time a model this size ships with day-zero support in vLLM, transformers, and llama.cpp. This Mistral Large 4 tutorial walks the whole arc for a beginner: the license rule that trips everyone up, the honest hardware math, the vLLM 8-GPU recipe, the llama.cpp GGUF route for Macs, and the API shortcut that is the sane first step for almost everyone. If you are here from the broader AI-models series, the Claude Haiku 5.5 explainer sets the landscape for the current frontier-model pricing era — this article is the hands-on counterpart for the open-weights side.

One orientation line before anything else. This guide is about the weights lane: downloading the Mistral Large 4 weights and running them yourself. It is not about the chat app — Le Chat, Mistral's consumer assistant, was rebranded to Vibe on May 28, 2026 — and it is not about small-model self-hosting on a laptop, which belongs to a completely different hardware class. The Mistral AI beginner guide covers the app and API lane; the self-hosted AI chatbot guide covers the small-model lane. Nothing below applies to either of those.

What Is Mistral Large 4, Exactly?

Mistral Large 4 is an open-weight, trillion-parameter mixture-of-experts language model from Mistral AI, released on October 29, 2026 with the model ID mistralai/Mistral-Large-4-1T-A52B. The "A52B" means about 49 billion parameters are active per token (52 billion including the embedding and output layers) — the model is a mixture of experts with 128 routed experts, four active per token, plus one shared expert, in a 72-layer decoder. It uses Multi-head Latent Attention with a DeepSeek-style sparse attention indexer, ships a 1.6-billion-parameter Pixtral vision encoder so it is natively multimodal (text and images in, text out), and supports a one-million-token context window. Reasoning effort can be set to low, medium, or high and defaults to high; it cannot be disabled. The model card recommends a sampling temperature of 1.0 and top_p of 0.95.

The codename matters only for search: Mistral's launch post is titled "Welcome Le Chonk," and the model card itself carries the "Le Chonk" title. When you see "Le Chonk tutorial" in search results, it means this model. And the name to put in API calls is mistral-large-4, with the API public preview having launched earlier, on October 6, 2026.

What Is the License Catch?

The license is the single most misunderstood part of this release, so it gets its own section up front: Mistral Large 4 ships under the Mistral Ecosystem Open-Weight License, which the model card describes as a modified version of the MIT license covering both commercial and non-commercial use — with the rule that operating a model-as-a-service business requires a separate commercial license with Mistral. The Hugging Face launch post phrases it the same way: an MIT derivative that establishes limitations for model-as-a-service offerings.

What that means in practice: you can download the weights, run them internally, fine-tune them, and build products on top of them. What you cannot do is rent out access to the model itself as a paid API or hosted endpoint business without going to Mistral for a commercial license first. This is the boundary between "open weights" and "fully open source," and it is exactly the line that decides whether your deployment is fine or needs a contract. If your plan is simply to run the model for your own team or your own agents, you are on the safe side of it.

What Is the Fastest Way to Run Mistral Large 4?

The fastest way to use Mistral Large 4 is the API; the fastest way to run the weights yourself is the vLLM Docker recipe below. Be honest about which one you need, because almost nobody reading this owns the hardware the weights require. Here is the complete beginner arc, in order:

1. Start with the API. Mistral's official inference pricing page lists Mistral Large 4 at $0.70 per million input tokens, $0.07 per million cached input tokens, and $2.10 per million output tokens — current as of publication; the price moved three times in one month (preview $0.68/$2.09, then list $1.36/$4.18, then the current rate), so re-check before budgeting. Call model ID mistral-large-4 and you are running the model in minutes, with zero hardware.
2. Size your hardware against the checkpoint you want. The official FP8 checkpoint is roughly 1.05 to 1.06 terabytes; the official NVFP4 checkpoint is roughly 610 gigabytes. Every official serving recipe assumes one node with eight GPUs and tensor parallelism of 8. If you do not have that, stop here and use the API or the GGUF route below.
3. Pick a route: vLLM for an 8-GPU node, llama.cpp GGUF for a 512 GB Mac. vLLM is the server-grade path; llama.cpp with a community GGUF is the accessible path. Both are covered below with their exact commands.
4. Run the recipe, then send one prompt. Each route ends with an OpenAI-compatible endpoint on port 8000. Your first prompt is the milestone; everything after is tuning.
5. Wire the agent flags if you are building an agent. Mistral Large 4 is positioned as a coding-agent model, and for agentic use the model card's --tool-call-parser mistral and --reasoning-parser mistral flags are required, not optional — the tool-call lesson below explains why.

That arc is the whole tutorial in miniature. The rest of this guide fills in each step with the numbers and commands you will actually need.

What Hardware Does Mistral Large 4 Need?

Mistral Large 4 hardware is the story of this release, and the honest version is: a data-center node or a maxed-out Mac, nothing in between. All official serving recipes assume one node with eight GPUs and tensor parallelism of 8 — the model card, the transformers recipe (torchrun --nproc-per-node 8), and the SGLang recipe (--tp-size 8) all agree on this. The original "8-GPU node or 512 GB Mac" framing checks out against the official sources, with one correction that matters: do not plan this around 8x80GB H100s. No official source names H100s, and an 8x80GB H100 node (640 GB total) does not fit the FP8 checkpoint at roughly 1.05 TB.

Two consequences fall out of this table. First, if your hardware is a single consumer GPU or a regular laptop, the weights routes are not for you yet — the API is the product. Second, the Mac route is real but narrow: it works because a 512 GB unified-memory machine can hold a 2-bit quantization, and even then you are running a heavily quantized model at modest generation speed. Neither of these facts is a secret, but no launch-coverage piece leads with them; now you have them before downloading a single byte.

How Do You Serve Mistral Large 4 With vLLM?

The Mistral Large 4 vLLM route is the server-grade one, and it has day-zero support: Mistral ships an official Docker recipe on the model card, so you never build anything from source. You need a Linux box with eight suitable GPUs, NVIDIA drivers, Docker, and the NVIDIA container runtime. Pull the image first:

docker pull vllm/vllm-openai:mistral-large-4

Then serve the FP8 checkpoint with the model card's exact command:

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 --ipc=host \
  vllm/vllm-openai:mistral-large-4 \
  --model mistralai/Mistral-Large-4-1T-A52B \
  --tensor-parallel-size 8 \
  --tool-call-parser mistral --reasoning-parser mistral \
  --enable-auto-tool-choice \
  --gpu-memory-utilization 0.85

The two parser flags are the lesson most beginners miss. Mistral Large 4 is a reasoning model that emits tool calls in Mistral's own format; without --tool-call-parser mistral and --reasoning-parser mistral, an agent framework talking to your endpoint will receive raw reasoning text where it expects structured tool calls, and the agent loop silently breaks. The Hugging Face launch post's NVFP4 recipe adds the expert-parallel flags for Blackwell nodes:

vllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --config-format mistral --load-format mistral \
  --tensor-parallel-size 8 \
  --decode-context-parallel-size 8 \
  --enable-expert-parallel

Two tuning details worth knowing. The checkpoint ships an embedded DSpark speculative-decoding draft head; enabling it with --speculative-config '{"method": "dspark", "num_speculative_tokens": 3}' is reported to raise throughput up to 1.9x. And on the quality side, the model card reports 48% on the Artificial Analysis Coding Agent Index — tied with GLM-5.3, using 47% fewer tokens per task — with the highest Intelligence Index score among western open-weight models at 26. Verify your endpoint the standard way: curl http://localhost:8000/v1/models should list the model, and then send your first chat completion.

Can You Run Mistral Large 4 as a GGUF?

The Mistral Large 4 GGUF route is the accessible one, with one important qualifier: the GGUF files are community builds, not official Mistral releases. The two official checkpoints are FP8 and NVFP4; the GGUFs come from the llama.cpp team's repository ggml-org/Mistral-Large-4-1T-A52B-GGUF (offering quantizations from IQ2_XXS and Q2_K up through Q4_K_M, NVFP4, Q8_0, and full BF16) and from Unsloth's builds. llama.cpp itself has day-zero support, and the Hugging Face launch post gives the official route. Install the runner:

curl -LsSf https://llama.app/install.sh | sh

Then serve a GGUF straight from the Hub — the launch post's example uses the 2-bit quant, which is the one that fits the 512 GB Mac:

llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K

Set expectations honestly. A 2-bit quantization of a trillion-parameter model is a serious quality compromise versus the FP8 weights; you are trading fidelity for the ability to run it at all on one machine. Community-reported sizing puts the 1-bit and 2-bit builds in the few-hundred-gigabyte range, but treat any exact number from a repo page as a community claim until you check the file list yourself. And if transformers is your framework rather than llama.cpp, the launch post's path is pip install --upgrade transformers with AutoModelForMultimodalLM and an eight-process torchrun — back to the 8-GPU node class, not the Mac.

Mistral Large 4 Weights vs API: Which Should You Pick?

This is the decision the whole guide has been building toward, so here it is as a table. The API and the weights are the same model; they are different products with different costs, and picking wrong wastes either money or weeks.

The framework comparison is short. vLLM is the production server: OpenAI-compatible API, tensor and expert parallelism, the parser flags, speculative decoding — everything above assumes it. llama.cpp is the accessible runner: one binary, GGUF quants, the Mac route, simpler to operate and far less demanding to set up. Neither replaces the other; they serve different hardware classes. And if your real need is a chatbot on a laptop or phone, you are in the wrong lane entirely — the self-hosted AI chatbot guide covers the small models that actually fit consumer hardware, which is a different world from a 1T-parameter mixture of experts.

What Are the Five Beginner Mistakes?

Every one of these comes from a real confusion beginners hit with this release. Read the table before you download anything.

Sources

• Mistral Large 4 model card — huggingface.co/mistralai/Mistral-Large-4-1T-A52B (release commit, architecture, vLLM recipe, license text, benchmarks, DSpark config)
• "Welcome Le Chonk" launch post — huggingface.co/blog/mistral-large4 (FP8/NVFP4 variants, day-zero support, NVFP4 recipe, llama.cpp + transformers recipes)
• Mistral inference pricing — docs.mistral.ai/inference/pricing ($0.70/$0.07/$2.10 per 1M tokens, current as of publication)
• "Mistral Large 4 Is Open. Running It on Macs Takes 338 to 590 GB" — medium.com (8-GPU vs 512 GB Mac analysis, M3 Ultra throughput figures)
• "vLLM Docker Setup: Mistral Large 4 in 13 Steps" — tech-insider.org (checkpoint sizes, FP8/NVFP4 hardware mapping)
• "Mistral Large 4 Open Weights Land: Enterprise Self-Hosting" — undercodetesting.com (4x B300 config, enterprise framing)
• "How to use Mistral Large 4: three ways in, and the hardware catch" — talkwithtech.com (API/weights/quantized routes, 4x B200/B300 for the quantized checkpoint)
• "How to Run Mistral Large 4 Locally" — apidog.com (8x H200 reference config)
• "Mistral Large 4 API pricing" — mercatus-ai.com (price history: preview $0.68/$2.09 → list $1.36/$4.18 → current $0.70/$2.10)
• "Mistral Large 4 open weights" release coverage — datanorth.ai (October 29, 2026 release corroboration)
• "How to Run Le Chonk" explainer — llms.blog (release + license + hardware explainer)
• "All-new Le Chat" announcement — mistral.ai (Le Chat app lane; rebranded to Vibe May 28, 2026)

Frequently Asked Questions

Is Mistral Large 4 free to use commercially?

Mostly yes, with one boundary. The Mistral Ecosystem Open-Weight License is a modified MIT license that covers commercial and non-commercial use — you can run the weights internally, fine-tune them, and build products on them. The exception is operating a model-as-a-service business, which requires a separate commercial license with Mistral. Running it for your own team or agents is on the safe side of that line.

How much VRAM does Mistral Large 4 need?

It depends on the checkpoint: the official FP8 checkpoint (~1.05–1.06 TB) fits 8x NVIDIA H200 (141 GB each); the official NVFP4 checkpoint (~610 GB) requires Blackwell GPUs such as B200/B300. A 2-bit community GGUF (~338 GB) fits a 512 GB Apple Silicon Ultra. Do not plan around 8x80GB H100s — no official source supports that config and the FP8 weights do not fit.

What does "Le Chonk" mean?

It is Mistral's own codename for the release — the model card is titled "Mistral Large 4 - Le Chonk" and the Hugging Face launch post is "Welcome Le Chonk." It has no technical meaning; it is just the name to search when you see "Le Chonk tutorial" in results.

Can I run Mistral Large 4 on a Mac?

Only the most powerful ones, and only with a heavily quantized community GGUF. A Q2_K quant of roughly 338 GB runs on an Apple Silicon Ultra with 512 GB of unified memory at about 90 tokens/sec prefill and 15 tokens/sec generation on an M3 Ultra. A regular MacBook cannot hold the weights — use the API instead.

Should I use the API or the weights?

Use the API first. At $0.70 per million input tokens and $2.10 per million output tokens (current as of publication, model ID mistral-large-4), you can run your first prompt in minutes with zero hardware. Commit to the weights only when you have the 8-GPU node or 512 GB Mac, need data to stay in your own infrastructure, or run enough volume that per-token pricing costs more than the hardware.