Releasesby GlyphManaged tables in Lakeflow pipelines (Beta)
You can now create managed tables in Lakeflow pipelines and write to them with one or more append flows. This feature is in Beta . You can also use import_checkpoint on an append flow to migrate an existing Structured Streaming workload into a pipeline without reprocessing the source. See create_table , CREATE TABLE ... FLOW (pipelines) , and append_flow .
Also shipped Sep 10, 2026Databricks in September 2026 →
BigQuery conversational analytics adds ML.CORRELATION in preview
Conversational analytics can now use ML.CORRELATION to calculate statistical correlations between a target column and one or more metric columns in a table.
BigQuery ML.METRICS evaluates predictions without a stored model
The preview ML.METRICS function computes classification or regression evaluation metrics on any table or query with actual and predicted values, with no need to create or reference a stored model.
You can use the AI.CAUSAL_EFFECT function to quantify the impact of specific interventions on time series data
You can use the AI.CAUSAL_EFFECT function to quantify the impact of specific interventions on time series data. This feature is in Preview .
BigQuery Data Engineering Agent integrates with BigQuery Graph
The Data Engineering Agent now uses BigQuery Graph for extra context between source and destination schemas, improving schema mapping accuracy in data engineering pipelines. It is generally available.
Using dbt artifacts for Slim CI and defer to production in dbt Projects on Snowflake (General availability)
Slim CI, defer to production, failed-execution recovery, and concurrent execution are now generally available for dbt Projects on Snowflake. Reuse dbt artifacts from recent production executions to accelerate development, streamline CI/CD workflows, and recover from failed production runs without maintaining a separate artifact store.
External secret providers (Preview)
External secret providers are now in preview. If your organization already keeps API keys, database passwords, and similar values in AWS Secrets Manager, Azure Key Vault, or Google Cloud Secret Manager, Snowflake can read those values when a query needs them, so you don't have to maintain a second copy in a Snowflake secret .
More from LakeflowEvery Lakeflow release →
Maintenance windows for continuous pipelines
You can now set a maintenance window on a continuous Lakeflow pipeline to control when Databricks applies platform updates that require a restart, such as new runtime versions and serverless infrastructure maintenance. Select a day of the week, a start hour, and a time zone, and Databricks starts maintenance only within that one-hour window.
Standard performance mode is now available for one-time runs in the Jobs API
You can now set performance_target to STANDARD in the Jobs API. Apache Airflow's DatabricksSubmitRunOperator also supports this. See Create and trigger a one-time run and Select a performance mode .
Session restore for serverless jobs is in Public Preview
Restore the Python variables and Spark session from a serverless compute job run into a new interactive notebook, so you can debug a failure or explore results without rerunning the entire job. Databricks captures the snapshot automatically for eligible runs. See Session restore for serverless jobs .
Celigo connector (Beta)
The Celigo connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector ingests audit log events, such as create, update, and delete operations on resources, from Celigo (integrator.io) into Databricks. Authentication uses a Celigo integrator.io API token.
Anysphere Organization connector (Beta)
The Anysphere Organization connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector uses the Anysphere (Cursor) Organization Admin API to ingest organization members, groups, and group members into Databricks as daily snapshots.
Anaplan connector (Beta)
The Anaplan connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector ingests audit trail events and user account records from your Anaplan tenant into Databricks. Authentication uses an Anaplan CA certificate and its paired private key. See Anaplan connector .
Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available
Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available. Today, Lakeflow Connect ingests formula fields by taking a full snapshot on each pipeline run by default. When incremental formula field ingestion is generally available, full snapshots will no longer be the default.
Zerobus Ingest into default storage is now in Public Preview
Ingesting into tables backed by default storage with Zerobus Ingest in Lakeflow Connect is now in Public Preview . See Create or identify the target table .