googledataproc-configure-connection-profile
This connection profile type is used only by Iceberg Writer. Some setup is required before creating the connection profile: see Create a new Dataproc cluster.
Connection Profile Name: Enter a name that uniquely identifies the external resource.
Namespace:.Select the namespace where the profile will be stored. All users who will use this profile must have READ and SELECT permissions on the namespace.
Endpoint Type: Select Google Dataproc.
Dataproc Cluster Name: specify the name of the Dataproc cluster you created as instructed in Create a new Dataproc cluster
Project ID: specify the project ID you used when you created the Dataproc cluster as instructed in Create a new Dataproc cluster
Region: specify the region of the Dataproc cluster you created as instructed in Create a new Dataproc cluster
Service Account Key: upload the service account key you downloaded as instructed in Create the service account and download the service account key; alternatively, this field may be left blank and the key path can be set via the environment variable GOOGLE_APPLICATION_CREDENTIALS (Striim must be restarted after setting the environment variable)
Connection Retry: See Client Side Retries > Client Library Retry Concepts
Additional Configuration: optionally specify Spark configuration properties, such as number of Spark executor instances or number of cores per executor, that will override the default Spark configuration