144 Apache Iceberg releases from data and AI companies, newest first
AWS Glue Data Catalog now supports vended credentials for Apache Iceberg REST catalog federation. This allows users to read external Iceberg tables using coarse-grained grants and temporary security credentials scoped to S3 prefixes.
Audit Explorer is a new governance application available in beta for searching and reviewing audit.3 logs. It features search and filtering by user or resource, activity charts, standardized log details, and session tracking to support security investigations and compliance reviews.
OneLake external-location support for Azure Databricks Unity Catalog is now generally available. This integration allows Unity Catalog-managed data to be stored directly in OneLake, providing a shared physical data foundation for both Databricks and Fabric.
Direct Iceberg access in AlloyDB for PostgreSQL is available for PostgreSQL 18 and later ( in Preview ). This feature lets you query Apache Iceberg tables in Cloud Storage or Lakehouse runtime catalog and accelerate analytical queries using the AlloyDB columnar engine. For more information, see Access Iceberg data from AlloyDB .
Amazon S3 Tables now supports managed migration of Apache Hive tables to Apache Iceberg. The service handles data validation, compatibility checks, type conversions, and file recompression as a managed, API-driven operation.
Amazon S3 Tables now provides managed migration of Apache Hive tables to Apache Iceberg, automating validation, conversion, and compaction while maintaining production workloads.
Snowflake announced general availability for backups of Snowflake Storage for Apache Iceberg tables and dynamic Iceberg tables hosted on AWS.
OneLake table APIs now support Iceberg REST Catalog writes, allowing engines like PyIceberg and DuckDB to commit data directly. A new metadata API in preview allows adding descriptions and tags to tables and columns.
Ray 2.60.0 released shuffle v2 as default for joins and aggregations, updated Parquet task sizing from row-group metadata, added rolling update surge controls in Ray Serve, and patched security vulnerabilities.
Lakehouse supports the following remote catalog providers for cross-cloud data access. This feature is generally available (GA). Databricks Unity Catalog on Amazon Web Services (AWS), Microsoft Azure, or Google Cloud AWS Glue Data Catalog on AWS Snowflake Horizon Catalog on AWS, Microsoft Azure, or Google Cloud Workday Data Lake on AWS or Google Cloud This...
With this release, you can read and write Apache Iceberg™ tables managed by another Snowflake account without creating, storing, or rotating secrets. This capability is generally available through catalog integrations for Snowflake Horizon Catalog. The accounts access the same table data without copying it.
Tri-Secret Secure is now generally available for Snowflake-managed Iceberg tables on AWS and Azure, enabling customer-managed encryption keys.
Automatic table optimization for Apache Iceberg is now available in the US West (N. California) region. It compacts small files and removes unreferenced snapshots to improve query performance.
Horizon Iceberg REST Catalog telemetry is generally available, logging external query engine HTTP API calls to Snowflake account event tables.
Sail, a Rust-native distributed compute framework, and LakeSail Cloud now connect to Microsoft OneLake. This integration allows teams to run Spark and Python workloads on Fabric using Delta Lake and Apache Iceberg formats without data copies or extra connectors.
Snowflake Cortex Code (CoCo) now supports generally available continuous and bulk file loading. Users can define file loading requirements to trigger Snowpipe pipelines for new files or COPY INTO commands for existing files, targeting standard or Iceberg tables.
The Lakebase snapshots and backup schedule APIs are now generally available. These APIs allow users to manage point-in-time snapshots and configure automated backup cadences for root branches.
AWS Glue Data Catalog now supports OAuth 2.0 bearer token authentication for Apache Iceberg REST Catalog APIs, allowing secure federated access for third-party analytics platforms.
Basin SQL is a serverless, distributed SQL engine for querying Apache Iceberg tables in Basin Catalog. It now supports OFFSET , which lets you skip a specified number of rows when retrieving query results. Use LIMIT to set the page size and OFFSET to skip earlier rows.
When you increase TARGET_FILE_SIZE on a Snowflake-managed Iceberg table, data compaction recompacts older files that were written under the smaller target and compacts new files using the larger setting. For example, if files were compacted at 16 MB and the new target file size is 128 MB, data compaction recompacts older files from 16 MB to 128 MB.
We're introducing Parquet Direct in preview. With Parquet Direct, you can create read-only Apache Iceberg™ tables in Snowflake from Apache Parquet™ files in your cloud object storage. Snowflake registers your existing files in place, so you can query your data without copying it or rewriting it, and with better query performance than external tables.
Foundry Worker data ingest and export, media ingest, and Iceberg extract operations are available without compute costs. This update aims to facilitate migration from Agent Worker to Foundry Worker. Streaming ingests will continue to be billed separately.
Transactions are generally available for both Unity Catalog managed Delta Lake tables and Unity Catalog managed Apache Iceberg tables. Group multiple SQL statements across one or more managed Delta Lake or managed Apache Iceberg tables into a single atomic transaction using BEGIN ATOMIC ... END; or BEGIN TRANSACTION; ... COMMIT; syntax.
Amazon S3, including S3 Tables and S3 Vectors, is now available natively on the Vercel Marketplace and v0. Developers can provision and connect object, vector, and open table storage directly within Vercel.
Metadata migration from Dataproc Metastore to Lakehouse is now generally available (GA). You can perform a one-time backfill migration or dry run to migrate metadata from a Dataproc Metastore service to an Apache Iceberg REST catalog endpoint or a Hive catalog endpoint using the Google Cloud console, the gcloud CLI, or the REST API.
You can now create and connect Amazon S3 resources through the Vercel Marketplace without leaving your Vercel project dashboard. The Amazon S3 Marketplace integration automatically adds connection details as environment variables and lets you manage resources alongside your application infrastructure, with links to AWS for deeper configuration.
We are thrilled to announce Unity Catalog 0.7.0 release, which includes exciting new features, many bug fixes, and improvements. A huge thank you to the awesome community who made this release possible!
Amazon EMR now validates data quality natively within Apache Spark jobs. Data engineering teams can catch quality issues inline before bad data reaches downstream systems, without building or maintaining separate validation infrastructure.
The following table types now support flexible column names by default: Iceberg external tables and Iceberg managed tables in BigQuery Apache Iceberg tables managed by the Lakehouse runtime catalog This feature is generally available (GA).
Autonomous AI Database can now use Apache Iceberg partition information to optimize queries on Iceberg external tables. Autonomous AI Database uses the Iceberg metadata to skip files that are not needed for a query.
Iceberg external tables and Iceberg managed tables now support flexible column names by default. This feature is generally available (GA).
Amazon Redshift now supports the creation and refresh of Apache Iceberg materialized views. Materialized views pre-compute expensive joins and aggregations once and store the results in an Apache Iceberg table in Amazon S3 or Amazon S3 table buckets, registered in the AWS Glue Data Catalog.
For installation instructions check out the getting started guide. Added postgres_cdc: Added incremental snapshot support to Postgres CDC, enabling tables to be backfilled in chunks while replication continues running, eliminating the need for snapshot-induced stream blocking or restart from the beginning.
Pipeline Builder now supports reading and writing Foundry Iceberg tables within Faster pipelines. This includes support for encrypted tables, data quality expectations, and overwrite or append operations.
AWS Glue Data Catalog now supports table optimization, statistics, and crawlers for Apache Iceberg Version 3 (V3) tables. With these new capabilities, you can automatically maintain V3 tables, optimize them for query performance, and discover them in Amazon S3.
Amazon Aurora PostgreSQL now supports direct queries of data stored in Apache Iceberg and Parquet formats within data lakes. This eliminates the need for ETL pipelines when combining transactional data with historical records.
Today, AWS announces system-managed materialized views for Apache Iceberg. Materialized views let you precompute an expensive query once and reuse the result across engines. Because this result is an open Iceberg table in your data lake, anyone with write access can change it, either through the catalog or by writing to the files in Amazon S3.
Amazon S3 Tables now supports all Apache Iceberg V3 data types and features, including deletion vectors and row lineage. Users can create new V3 tables or upgrade existing V2 tables to improve performance and storage efficiency.
With this Public Preview release, you can use ATTACH MASKING POLICY in DCM Projects definition files to declaratively assign a masking policy to one or more columns in tables, Apache Iceberg™ tables, views, or dynamic tables. DCM Projects manages the masking policy attachment throughout its deployment lifecycle.
Amazon SageMaker Unified Studio now supports two new connection capabilities: (1) Iceberg REST Catalog (IRC) connections for external Apache Iceberg catalogs, and (2) IAM authentication for Amazon DocumentDB. Together, they let data teams connect to a broader set of governed data sources and use credential-less authentication across projects.
You can now write the output rows produced by BigQuery continuous queries directly into Apache Iceberg managed tables . This lets you continuously process streaming data from BigQuery and write it into your open-format lakehouse by using an INSERT DML statement .
With this release, hybrid tables support CHECK constraints . Previously, CHECK constraints were limited to standard tables and Snowflake-managed Iceberg tables, so a hybrid table workload had to enforce value-level rules in application code.
On-Demand Compute lets eligible, compute-intensive SELECT queries tap additional ClickHouse workers through your existing service and endpoint, no resizing or extra provisioning required.
Amazon EMR 7.14 is now available with new features across Amazon EMR on EC2, Amazon EMR on EKS, and Amazon EMR Serverless, along with version upgrades for additional applications. This release also upgrades Apache Spark to 3.5.8 and Apache Iceberg to 1.10.1.
With this release, we're pleased to announce the general availability of partition evolution for Snowflake-managed Iceberg tables. You can now change a table's partition specification after you create the table by adding, dropping, or replacing partition fields, without rewriting existing data.
For installation instructions check out the getting started guide . Added oracledb_cdc: Oracle CDC connector now includes username as metadata in events for better traceability. ( @josephwoodward , #4674 ) Fixed iceberg: Replaced MinIO with silo fork in tests to work around MinIO's removal from Docker Hub.
Today, we are excited to announce the General Availability of New Relic Lens , a powerful data federation capability designed to simplify and accelerate data discovery and analysis across multiple environments.
Lake Compute is a dedicated engine built on DuckDB that runs dbt models directly against Apache Iceberg tables. It is priced specifically for transformation.
Managed Iceberg tables and Palantir's Iceberg REST catalog are GA on standard Foundry enterprise environments on AWS, Azure, and GCP, with Palantir-managed or customer-provided storage.
Elastic Channels for Snowpipe Streaming are now generally available. Elastic Channels provide one implicit, server-scaled ingestion path per pipe, so concurrent producers can stream directly to Snowflake tables and supported Snowflake-managed Iceberg tables without managing channel names, lifecycle, ordering, or offset tokens.
Automatic change data feed (Auto CDF) is rolling out to workspaces over the next month and will be available in all supported regions by the end of October 2026. Auto CDF computes row-level changes at query time using row tracking, so you don't need to enable change data feed on each Delta Lake or Apache Iceberg v3 table.
Sharing and reading foreign Iceberg tables with OpenSharing is now generally available. Providers can share Iceberg tables federated from foreign Iceberg catalogs using Lakehouse Federation, including to recipients using external Iceberg clients.
Public Preview release of Iceberg Scan Plan API in Horizon Catalog to enforce data protection policies.
For installation instructions check out the getting started guide . Added avro: Added max_decompressed_block_bytes configuration field to cap OCF block decompression and prevent decompression-bomb denial-of-service attacks.
Lake Compute is a dedicated engine built on DuckDB that runs dbt models directly against Apache Iceberg tables, priced for transformation only. It is in private beta.
Automatic change data feed (Auto CDF) computes row-level changes at query time using row tracking, without requiring you to manually enable change data feed on each Delta Lake or Apache Iceberg v3 table. Auto CDF removes write-time overhead and makes MERGE and UPDATE operations about 15% faster on tables that you query for changes.
The $iceberg stage writes to AWS S3 using s3:// instead of s3a://, increasing compatibility with external query engines.
Today, we are announcing the general availability of AWS Glue 6.0 , delivering 30% lower pricing than previous AWS Glue versions and introducing full support for Apache Iceberg v3 features. AWS Glue 6.0 is built on a fully modernized runtime, Apache Spark 4.1, Python 3.13, and Scala 2.13, delivering faster performance.
You can now use the Horizon Iceberg REST Catalog (IRC) API to access externally managed Apache Iceberg™ tables in a catalog-linked database from external engines such as Apache Spark™, Trino, DuckDB, or PyIceberg.
Sharing managed Iceberg tables in Databricks-to-Databricks sharing and open sharing is now generally available. Providers can now also share managed Iceberg tables to external Iceberg clients. See Add managed Iceberg tables to a share and Read shared managed Iceberg tables .
Releasesby Glyph