There Is No Off-the-Shelf Stack for Regulated AI

The market wants to sell you a stack. Pick a provider, adopt their models, deploy in their region, and you are running agents by the end of the quarter. For a lot of businesses, that is a perfectly good answer.

However, in a regulated enterprise it is a trap, and the reason is not commercial. Which models you are permitted to use, and where they are physically allowed to run, is decided by your regulatory regime and your own security homologation, not by a vendor's default. An off-the-shelf stack encodes someone else's answer to a question only your compliance function can answer. That is why it never passes review.

A bank under DORA and data-residency obligations needs a fundamentally different deployment than a hospital, and both need something different from a vendor's reference architecture. The stack is not a product you buy. It is a set of decisions you have to own.

Lock-in is a concentration risk, not just a commercial one

Standardizing on a single model provider feels like prudent simplification. In a regulated context it quietly creates a risk your supervisor actively cares about.

The model layer is the fastest-moving part of the whole stack. Bind your architecture to one provider and your architecture ages at that provider's pace, inherits that provider's outages, and depends on that provider's continued willingness to serve you on acceptable terms. In an unregulated business that is a commercial nuisance. Under an operational-resilience regime, dependence on a single unreplaceable third party is a concentration risk, and demonstrating that you are not exposed to one is close to an obligation.

So provider-agnosticism is not a nice-to-have you add for negotiating leverage. It is evidence your compliance function has to be able to produce: here is how we route around any single provider, and here is the tested path we would take if one became unusable tomorrow.

The model gateway makes providers interchangeable

The answer to lock-in is a single internal interface every agent calls, behind which the models are interchangeable. Anthropic, Mistral, or an open-source model running on your own GPUs all sit behind the same door. The agent does not know or care which one answered.

Three properties separate a real gateway from a decorative one.

A documented, tested fallback. If a provider is unavailable or fails a policy check, the gateway routes to an alternative automatically. Portability that has not been tested before you depend on it is a slogan, not a control. Under an operational-resilience regime, an automatic, tested fallback is close to a requirement.

At least three portable options documented before go-live. Not after. The time to prove you can move is while you still have the option not to. Three documented, substitutable options before you depend on any of them is what turns "we could switch" into "we have switched, here is the runbook."

Routing by sensitivity, not convenience. The gateway decides where a workload may run based on the data it carries, not on which provider is cheapest or fastest today. That decision belongs to the next section, because it is the one regulators scrutinize hardest.

Deploy topology is a data-sovereignty decision, routed per class

Here is the part most reference architectures get wrong: they pick one deployment topology for everything. That is how you end up either over-constraining cheap, non-sensitive work, or, far worse, letting regulated data reach a place it should never have gone.

Topology is not one decision. It is a routing rule. Data is classified by sensitivity, and the classification decides the topology.

The rule sits above the table: classify the data, then the table tells you where that class may run. A single business process may itself contain both sensitive and non-sensitive steps, so the routing happens per step, not once for the whole system.

The posture that makes this defensible is sovereignty routing that fails closed. Regulated and PII data stays inside your perimeter by default. Only explicitly non-sensitive workloads may reach an external-region model, and only if you have allowed it. When a classification is uncertain, the system treats the data as sensitive and keeps it in. The default protects you, and it takes a deliberate, logged decision to relax it. "Your data never leaves your walls" is then not a marketing line. It is an architecture in which egress is denied by default and permitted only per policy.

Owning the stack means owning the exit

Provider-agnostic and deploy-anywhere only hold if you can prove them under pressure, which means the exit has to be real, not rhetorical.

Two things keep an inherited platform from quietly becoming the lock-in you were trying to avoid. Both belong in the contract, in writing.

Escrow and a tested exit. Source and weights in escrow, and a documented, tested exit plan. A migration path you have never rehearsed is a wish, not a guarantee.

You own the asset. The governed runtime and the agents are yours to operate. Your roadmap keeps moving when the implementation partner walks away. A serious partner holds itself to being a replaceable third party under your regulators' rules, because that is exactly what those rules require of your critical suppliers.

The point is not that you must build every layer yourself. Rebuilding a model gateway, a policy engine, and a deployment fabric from zero is roughly a year of hard plumbing, most of it in the security and compliance details a regulator inspects. The point is that whoever builds it, the decisions stay yours: which models, run where, moved how. Those are compliance decisions wearing engineering clothes.

So the question is not "which stack should we adopt?" It is if our primary model provider became unusable tomorrow, for a commercial, regulatory, or outage reason, what is the tested path to the next one, and is it documented before go-live? If you cannot answer that per model and per data class, you do not have a deployment decision yet. You have a deployment hope.

If you want to map your data classes to permitted topologies on a real process, book a meeting.