Anyscale, August 2026 new features

Ray-2.58.0

Highlights Ray Serve LLM: In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the LLMRouter ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress...

Ray

Highlights Ray Data: In this Ray release, we've enabled DataSourceV2 by default ( #64821 ), so read_parquet and friends use the new scan/listing infrastructure with row-group-aware chunking and predicate splitting. Hash Shuffle V2 eliminates the aggregator actor pool.

• Ray-2.56.1
• Ray-2.57.0