Amazon SageMaker HyperPod adds Global Inference Gateway: one endpoint routes inference across clusters and regions with automatic failover
Announced November 20, 2026
Verified (fact)
AWS introduced Global Inference Gateway, a managed single endpoint for SageMaker HyperPod. It sends inference requests to HyperPod clusters within one region or across multiple regions, selects the most suitable cluster, and automatically fails over to another cluster when one goes down. All requests are authenticated with SigV4 and IAM over AWS PrivateLink.
Why it matters (interpretation)
Teams serving models on their own clusters no longer need to build cross-region routing and failover themselves, easing the burden of securing GPU capacity and managing availability.
Sources
More about AWS
-
AWS Quick update: build and run process flows from chat, query multiple agents at once, new Meta Ads and Amplitude connectors
AWS announced new features for AWS Quick, its AI workspace for business use. Users can create and immediately run flows for business...
-
AWS releases agent skill to set up distributed KV cache for self-hosted LLMs (claims up to 40% lower cost, 80% lower TTFT); adds HyperPod Slurm extensions
AWS added a distributed KV caching skill to the Agent Toolkit for AWS. With this skill, a coding agent decides whether to use a...
-
Amazon Textract Insights is generally available: describe fields in natural language, get structured values with confidence scores and page-level grounding
AWS made Textract Insights, a generative AI feature of Amazon Textract, generally available. Users describe the fields they need in...
-
AWS customer case: Standard Bank Group automates generative AI use-case risk assessment with LLM-as-a-judge on Bedrock
Standard Bank Group worked with AWS Professional Services to automate the preparation of supporting evidence for its generative AI...
-
Amazon Bedrock AgentCore payments adds AWS Marketplace support, letting agents buy x402 products billed in dollars to AWS bills
According to AWS, Bedrock AgentCore payments now works with AWS Marketplace. Agents can find and buy Marketplace products (APIs, data,...