Feature summary
Category | Details |
|---|---|
Supported target path | Google Cloud lakehouse: GCS data lake, GCP Managed Apache Spark compute, GCP Lakehouse Runtime Catalog, and GCS external staging. |
Target adapter | GCPLakehouseWriter |
Flow Designer component | Google Cloud Lakehouse, under Data Warehouses |
Supported workloads | Initial load and CDC workloads from Striim-supported sources. |
Supported write modes | APPENDONLY for initial snapshot loads; MERGE for CDC workloads. |
Catalog endpoint types | IcebergRESTCatalog and CustomIcebergCatalogForBigQuery. |
Recommended catalog mode | REST catalog mode for new deployments when prerequisites are met. |
Credential vending | Supported only in REST catalog mode when the Google Cloud catalog is configured for credential vending. |
Schema creation | Supported during initial load when the source sends schema creation events. For database initial loads, enable schema creation on the source. |
Schema Evolution | Supported for selected source DDL operations, including namespace and table creation, add/modify/drop column, drop table, and truncate table. Unsupported DDL operations halt the application. |
Batching | Controlled by UploadPolicy, with eventcount and interval sub-policies. |
Parallel initial load | ParallelThreads can accelerate APPENDONLY initial loads when the Spark cluster has enough executor capacity. |
Optimized merge | OptimizedMerge supports partial-image CDC events in MERGE mode when events include updated fields and key fields. |
Recovery | At-least-once delivery. In APPENDONLY mode, duplicate rows may appear after restart because reprocessed events are not de-duplicated. |
Key limitations | REST catalog mode requires a bucket-root IcebergTablesLocation; partitioned tables are not created automatically; Spark writes Iceberg TIME values as strings; in MERGE mode, the only supported special character in table names is underscore (_). |