Releasesby GlyphYou can now pause a Dataflow batch job using the pause_on_failure service option
You can now pause a Dataflow batch job using the pause_on_failure service option. This feature lets you preserve the state of your batch pipeline job, address external issues, and resume processing without losing completed work. You can use this option to automatically pause a job on failure, or use this feature to manually pause a job when you chose to.
Also shipped Jul 31, 2026Google Cloud in July 2026 →
Web terminal on serverless GPU compute (AI Runtime) is in Public Preview
The web terminal in notebooks connected to serverless GPU compute (AI Runtime) is now in Public Preview . Use it to run shell commands, monitor GPU usage with nvidia-smi , install libraries, and manage files directly on the compute. This requires serverless GPU environment version 5 or above. See Run shell commands in Databricks web terminal .
Packaged clean rooms are now generally available
Packaged clean rooms are now generally available. In a packaged clean room, a provider supplies notebooks, JARs, and data that a consumer runs against their own data without seeing the provider's code or data assets. See Packaged clean rooms .
OpenAI connector (Beta)
The OpenAI connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector uses the OpenAI Organization Admin API to ingest organization administration data, such as users, projects, API keys, usage, costs, and audit logs, from OpenAI into Databricks.
[In preview] Public Preview: Azure SQL updates for late-July
In late-July 2026, the following update and enhancement was made to Azure SQL: Regex-based dynamic data masking (DDM) extends the built-in DDM feature with pattern‑based masking to hide sensitive data like emails, phone numbers, and identifiers, centrally
Upcoming August 2026 model deprecations in GitHub Copilot
We will deprecate the following models across all GitHub Copilot experiences (including Copilot Chat, inline edits, ask and agent modes, and code completions), on September 1st, 2026: Model Deprecation date Suggested alternative Gemini 3.1 Pro 9-1-2026 Gemini 3.6 Flash Claude Opus 4.5 9-1-2026 Claude Opus 4.7, Claude Opus 4.8, or Claude Opus 5 Claude Opus...
Gemini 2.5 Pro and Gemini 3 Flash deprecated
As of today, July 31, 2026, we have deprecated the following models across all GitHub Copilot experiences (including Copilot Chat, inline edits, ask and agent modes, and code completions).
More from DataflowEvery Dataflow release →
You can now use Hyperdisk Balanced disks for Dataflow worker VMs
You can now use Hyperdisk Balanced disks for Dataflow worker VMs. With Hyperdisk Balanced disks, you can provision IOPS and throughput independently of disk size by using the diskProvisionedIOPS and diskProvisionedThroughput pipeline options (Java SDK) or disk_provisioned_iops and disk_provisioned_throughput_mibps pipeline options (Python and Go SDKs).
Dataflow now supports NVIDIA RTX Pro 6000 GPUs
Dataflow now supports NVIDIA RTX Pro 6000 GPUs. You can use this GPU model to run your Apache Beam pipelines on Dataflow. RTX Pro 6000 GPUs are recommended for large, medium, and small model inference workloads. To configure your workers with this GPU model, set the accelerator type to nvidia-rtx-pro-6000 .
Dataflow has updated and expanded its pipeline update features for streaming jobs: - Automated...
Dataflow has updated and expanded its pipeline update features for streaming jobs: Automated stop-and-replace updates : You can perform automated, declarative stop-and-replace updates to streaming jobs. Parallel updates with the same job name : When you perform automated parallel updates, you can use the same job name for the new replacement job.
Dataflow Auto VM Selection now supports 4th generation Compute Engine machine series
Dataflow Auto VM Selection now supports 4th generation Compute Engine machine series. When Auto VM Selection is enabled, the service prioritizes these newer generation VMs for improved performance.
Dataflow now supports streaming high availability for Streaming Engine jobs ( Preview )
Dataflow now supports streaming high availability for Streaming Engine jobs ( Preview ). When you enable the requires_high_availability service option, Dataflow replicates backend state across multiple zones in a region to protect your job from single-zone failures. For more information, see Streaming high availability .
Dataflow job builder now supports external Iceberg REST Catalogs as a source
Dataflow job builder now supports external Iceberg REST Catalogs as a source. You can now ingest data from external Apache Iceberg REST catalogs (IRC) directly into Lakehouse tables using Dataflow's job builder UI without writing code. For more information, see Import data from external Iceberg catalogs to Lakehouse using Dataflow .
When you use min_ram or cpu_count resource hints for pipeline steps that don't require accelerators, Auto VM...
When you use min_ram or cpu_count resource hints for pipeline steps that don't require accelerators, Auto VM Selection (Instance Flexibility) is enabled automatically. With Auto VM Selection, workers are provisioned from a curated list of machine types that meet your RAM and CPU requirements.
Dataflow support for the C4A machine series of Arm processors is now generally available
Dataflow support for the C4A machine series of Arm processors is now generally available. Arm-based VMs are optimized for power efficiency and can provide improved price-performance for many workloads. For more information, see Use Arm VMs on Dataflow .