Bedrock AgentCore Gateway adds InvokeModel APIs and automatic model fallback routing

AgentCore Gateway started as a governance layer for agent tool calls. With this update it also fronts model inference. Clients call the standard Bedrock InvokeModel or streaming API, and the gateway applies policy, logging and routing centrally, with no SDK rewrites required.

The headline feature is fallback. Teams define an ordered chain of providers or models. If the primary returns throttling or capacity errors, the gateway moves to the next. Built-in circuit breakers stop sending traffic to an endpoint that keeps failing, which reduces latency spikes and avoids retry storms during outages.

This is AWS absorbing a pattern that had been the territory of third-party gateways such as LiteLLM, Portkey and OpenRouter. The timing is notable: this week a compromised LiteLLM 1.107.0 release on PyPI shipped a credential stealer and backdoor, an HN story with 588 points. Security-minded teams now have a stronger reason to prefer a managed gateway over a self-hosted proxy in the request path.

Caveats: fallback across providers can change model behavior mid-session, so outputs may differ in format or quality when traffic fails over. Teams need evaluation coverage for every model in the chain. Watch for how cost attribution and guardrails apply consistently when a request lands on a secondary provider.