Data and AI releases

2,002 this month · 180 companies

Amazon SageMaker AI now supports KV cache-aware routing for real-time inference endpoints

UpdateNot yet verified

Amazon SageMaker AI now supports KV cache-aware routing for real-time inference endpoints. Working with engines like vLLM, SageMaker routes requests to endpoint instances holding matching prompt caches, reducing time to first token for workloads with shared system prompts or chat history.

LLMsPerformanceDeveloper tools

Read on aws.amazon.com

Also shipped Nov 16, 2026AWS in November 2026 →

More from Amazon SageMakerEvery Amazon SageMaker release →