Global inference routing

Mixlayer Team·October 6, 2026

Mixlayer now let's you launch dedicated inference capacity globally and routes your requests automatically.

By default, nothing changes for you: requests to https://mixlayer.ai/v1 enter our network at the nearest edge and run in the closest region that serves the model you asked for. If you need more control, because of latency, compliance or a backup plan, you can now decide where inference happens.

Two regions per request

Every request has a gateway region, where it enters Mixlayer's network, and an inference region, where the model actually runs. They're often the same region, but they don't have to be.

To choose the entry point, use a regional hostname such as https://us-central.mixlayer.ai/v1. It takes the same paths and the same keys. A regional hostname picks the gateway only. Your organization's routing policy still decides where inference runs.

A request enters at a gateway region (the nearest edge, or the one named in a regional hostname). Your routing policy then picks the inference region. Fallback happens only before inference starts, and only to regions the policy allows. If none can serve the model, the request fails with a 503 rather than widening your policy.

Routing policies

A routing policy is set once for the whole organization through the Platform API. It has three fields:

mode decides how a region is picked: nearest (latency-based, the default), ordered (your priority list) or required (each request must name a region).
fallback decides where a request can go if the first region can't serve the model: all (the default), same_country or disabled.
regions limits inference to the region IDs you list. Leave it empty to allow every region visible to your organization.

Some common setups:

Primary and backup. Use mode: "ordered" with regions: ["us-central", "us-east"] to always try us-central first and fall back to us-east when the model isn't available there. Regions outside the list are never used.

Stay in one country. Use fallback: "same_country" to allow failover while keeping inference in the country of the first selected region. To pin the country itself, list only that country's regions.

One region, no exceptions. Use mode: "ordered", fallback: "disabled" and a single region. If the model can't be served there, the request fails rather than going elsewhere.

Per-request control

Send an Mxl-Region header to try a specific inference region first. The policy still applies, so the header can't send traffic to a region you've excluded. HTTP responses include an Mxl-Inference-Region header showing where the request actually ran, including for streaming and embeddings.

What fallback does and doesn't do

Fallback happens when the model isn't available in a region at the start of inference. Once generation starts, the request stays in that region. We don't restart a half-finished generation somewhere else, and we don't retry timeouts or model errors in another region. If no eligible region can serve the model, you get a 503 with no_model_server_available instead of a quiet widening of your policy.

Model availability varies by region. To see which regions serve each model for your organization, list your models through the Platform API and check each model's regions array.

The regional routing guide has every option with copy-paste examples.