Intermediate · 10 minutes

How to Deploy and Query Xiaomi MiMo-V2.6 Pro (vLLM FP8 & OpenRouter API)

A complete developer guide on self-hosting Xiaomi’s 1.02T parameter MiMo-V2.6 Pro model using vLLM FP8 tensor parallelism or querying it globally via the OpenRouter API.

Step 1: Choose Deployment Method: Cloud API vs. Self-Hosted Cluster

For production agent prototypes with zero server management, connect to OpenRouter’s endpoint at "xiaomi/mimo-v2.6-pro" for $0.10/M input tokens. For sovereign enterprise hosting, deploy the MIT-licensed Hugging Face weights to an 8x H100 (80GB) node.

# Check GPU memory and CUDA compatibility on local node
nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv

Step 2: Option A: Query MiMo-V2.6 via OpenRouter API

Configure an OpenAI-compatible SDK client pointing to OpenRouter’s base URL. MiMo-V2.6 Pro supports up to 128k context and structured JSON tool calling.

import OpenAI from "openai";

const openRouterClient = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
  defaultHeaders: {
    "HTTP-Referer": "https://aiintoai.vercel.app",
    "X-Title": "AIxAI Intelligence Studio"
  }
});

export async function auditRepositoryCode(diffPatch: string) {
  const response = await openRouterClient.chat.completions.create({
    model: "xiaomi/mimo-v2.6-pro",
    messages: [
      { role: "system", content: "You are an automated cybersecurity verifier trained on CyberGym. Detect vulnerabilities and generate fix patches." },
      { role: "user", content: diffPatch }
    ],
    temperature: 0.1
  });

  return response.choices[0].message.content;
}

Step 3: Option B: Launch vLLM Local Server with FP8 Quantization

Install the latest vLLM release supporting MiMo’s Multi-Head Latent Attention (MLA) and fine-grained 128-expert routing. Launch the OpenAI-compatible server using 8-way tensor parallelism.

python3 -m vllm.entrypoints.openai.api_server \
  --model XiaomiMiMo/MiMo-V2.6-Pro \
  --tensor-parallel-size 8 \
  --quantization fp8 \
  --max-model-len 131072 \
  --trust-remote-code \
  --port 8000

Step 4: Download and Test MiMo RL Environments (Optional)

If training or validating custom reward models, clone the official reinforcement learning gym environments released by Xiaomi under the MIT license.

git clone https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL
cd MiMo-V2.6-Flash-RL && pip install -r requirements.txt
python verify_trajectory.py --env swe_bench_verified