Releasesby IbisDataFrame caching on serverless compute is now in Public Preview
PreviewyesterdayUnverifiedAdded Oct 23, 2026
DataFrame caching on serverless compute is now in Public Preview . You can cache a DataFrame with df.cache() or df.persist() on serverless compute to reuse an expensive result for the rest of your session.
More Databricks Data Intelligence Platform releases
Every Databricks Data Intelligence Platform releaseDatabricks Runtime 19: October 19, 2026
Behavioral changes Spark Connect DataFrame execution metrics now hide internal operators and metrics associated with row filters and column masks. This protects information about policy execution when inspecting df.executionInfo.metrics , including its text representation. The boundary of the protected portion of the plan and its own metrics remain visible.
Filter data per end user with OAuth custom identity claims
You can now add a custom identity claim to a service principal's OAuth token and filter rows by it in a dynamic view or ABAC policy. Serve many end users through one service principal while each sees only their own data. See Filter data per end user with OAuth custom identity claims .
DeepSeek V4.1 Flash is scheduled for retirement
DeepSeek V4.1 Flash will be retired on November 19, 2026. DeepSeek V4.1 Flash (1017) is the recommended replacement. See Deprecated and retired models for migration guidance.
ADBC driver for Power BI is now generally available
The Arrow Database Connectivity (ADBC) driver for the Databricks connector for Power BI is now generally available. ADBC is the default driver for new connections in Power BI Desktop and the Power BI service. See Configure ADBC or ODBC driver for Power BI .
On-demand state repartitioning is now generally available
On-demand state repartitioning is now generally available. You can resize the number of shuffle partitions for a stateful Structured Streaming query or Lakeflow pipeline without losing checkpoint state. See On-demand state repartitioning for stateful streaming queries .
AI Runtime will soon be available by default for workspaces with the compliance security profile enabled
AI Runtime is scheduled to become generally available on October 15, 2026, on AWS and Azure for workspaces without the compliance security profile enabled. AI Runtime will be available by default for eligible workspaces with the compliance security profile enabled in mid November. Availability depends on the supported compliance standards and regions.
Databricks AI Runtime SDK replaces serverless_gpu
Starting in GPU environment version 6, the Databricks AI Runtime SDK ( databricks-sdk-air ) replaces the deprecated serverless_gpu package. It provides the @distributed and ray_init APIs and the databricks.air.data utilities under the databricks.air namespace. The gpus and gpu_type arguments are renamed to num_accelerators and accelerator_type .
Feature Views will soon be available by default for workspaces with the compliance security profile enabled
Feature Views are now generally available. They will be automatically enabled for workspaces with the compliance security profile enabled in mid-November 2026. Define and compute features declaratively from batch or streaming data, then use them for model training and online serving. See Feature Views .
Also shipped on Oct 23, 2026
Databricks in October 2026Retrieve Cortex Search (Public Preview)
Retrieve Cortex Search is now available in Public Preview. The CORTEX_SEARCH_RETRIEVE table function lets you run low-latency Cortex Search queries directly from SQL, either one query at a time or one query per input row with a LATERAL join. Use it for real-time workloads such as retrieval-augmented generation, chatbots, and in-app search.
GitHub Copilot in Visual Studio , September update
September 2026 brought deeper visibility into your work: explore pull requests without leaving chat, bring the AI model your team trusts, and fix vulnerable packages the moment Visual Studio flags them. Highlights Here’s what’s new with GitHub Copilot in Visual Studio 2026.
Flagship - Target Flagship feature flags with structured evaluation context
Flagship supports nested objects, arrays, and null in evaluation context. Pass user profiles and tag lists directly, then target nested attributes without flattening your application data. For example, pass a profile and tags when evaluating a flag through the Workers binding: const enabled = await env. FLAGS .
Responses API for the Cortex REST API (Preview)
You can now call OpenAI models through the OpenAI-compatible Responses API, in preview. Send requests to /api/v2/cortex/v1/responses on the Cortex REST API, or to /v1/responses on the Cortex AI Gateway, with the OpenAI SDK or any other client that uses the Responses API.
Manage Test Engine workflows with Terraform and the REST API
You can now manage Test Engine workflows with Terraform and the REST API instead of configuring each one in the UI. The new buildkite_test_suite_workflow resource in Buildkite Terraform provider v1.43.0 lets you define monitors, filters, and actions in code, and import existing workflows.
arize-phoenix: v20.25.0
20.25.0 (2026-10-22) Features factory: scaffold managed agents issue-to-PR factory (e5e360b) Bug Fixes factory: keep claude-lock.json on a factory-state branch instead of main (2e131a8) sandbox: keep teardown alive on cancellation (2779b46) server: format the ID in the experiments 404 error details (33ed20a) server: work around graphql-core 3.3.0 subfield...