Striim Platform 5.0 documentation

WAITING_TO_PROCESS: Database Reader had not read the table before the application was interrupted.

If Create Schema is True, you must manually drop the table before restarting the application.

If Create Schema is False, you must manually truncate the target table before restarting the application.

Striim will drop and re-create the target table.

Striim will attempt to truncate the target table. If the table does not exist, the application will halt.

SCHEMACREATION_COMPLETED: Create Schema is True and the table has been created in the target, but Database Reader had not started reading the table's data before the application was interrupted.

You must manually truncate the target table before restarting the application. (Writing data may have begun even though the status had not yet changed to IL_INPROGRESS.)

Striim will drop and re-create the target table

Striim will truncate the empty target table.

IL_INPROGRESS: Database Reader was reading the table when the application was interrupted.

You must manually truncate the target table before restarting the application.

Striim will drop and re-create the target table

Striim will attempt to truncate the target table. If the table does not exist, the application will halt.

IL_COMPLETED: The table was fully written to the target before the application was interrupted.

No action required.

Database Reader will not read the table again.

Database Reader will not read the table again.

Connection Profile Name

enum

Appears in Flow Designer only when Database Provider Type is BigQuery and Use Connection Profile is True. See Introducing connection profiles.

Connection URL

String

Create Schema

Boolean

False

If set to True, when Database Reader's output stream is the input stream of an Azure Synapse Writer, BigQuery Writer, Database Writer (for MariaDB, MariaDB Xpand, MySQL, Oracle, PostgreSQL, SQL Server, or YugabyteDB only), Databricks Writer, Snowflake Writer, or Spanner Writer target, the schema(s) and tables specified in the Tables property in the writer will be created in the target.

For example, if the Tables property in the writer is sourceschema1.%,targetschema1.%;sourceschema2.%,targetschema2.% then the schemas targetschema1 and targetschema2 will be created in the target, and all the tables in sourceschema1 and sourceschema2 will be created in the corresponding target schema.

Data types will be mapped as per Data type support & mapping for schema conversion & evolution. Single-column PRIMARY KEY constraints will be replicated in the target. DEFAULT, FOREIGN KEY, NOT NULL, composite PRIMARY KEY, and UNIQUE KEY constraints will not be replicated.

If a table already exists in the target, Striim will verify that the source and target structure match. If they do not, the application will halt and log a TargetTableSchemaMismatchException.

MON output for the target tables will include a schemaCreationStatus of Pending, InProgress, Success, or Failed.

Database Provider Type

String

Default

Controls which icon appears in the Flow Designer and whether Use Connection Profile appears in the web UI.

Excluded Tables

String

Data for any tables specified here will not be returned. For example, if Tables uses a wildcard, data from any tables specified here will be omitted. Multiple table names (separated by semicolons) and wildcards may be used exactly as for Tables.

Fetch Size

Integer

100

Sets the maximum number of records to be fetched from the database in a single JDBC method execution (see the discussion of fetchsize in the documentation for your JDBC driver).

JAAS Configuration

String

If you Enable Kerberos authentication, use this property to configure it.

Password

encrypted password

The password for the specified user. See Encrypted passwords.

Parallel Threads

Integer

1

For Fast Snapshot Loading to Azure Synapse, BiqQuery, Databricks, Fabric Data Warehouse, Microsoft Dataverse, or Snowflake, specify the number of threads Database Reader will use. This value should not be higher than the number of tables to be read. For other targets, leave set to the default of 1.

When Database Reader is started, the tables will be distributed among the threads in round-robin fashion, then each thread will read one table at a time. When the Tables property is a list of tables, the tables are read in that order. If the Tables property uses a wildcard, the sequence is determined by the order in which the JDBC driver gives Database Reader the list of tables.

Known issue DEV-49013: the maximum number of tables supported by Parallel Threads is 255.

Query

String

Optionally, specify a single SQL SELECT statement specifying the data to return. You may query tables, aliases, synonyms, and views.

Query is not supported when Create Schema is True, Parallel Threads is greater than 1, or Restart Behavior on IL Interruption is Truncate_target_table or Replace_target_table.

When Query is specified and Tables is not, the WAEvent TableName metadata field value will be QUERY. When both Query and Tables are specified, the data specified by Query will be returned, and the Tables setting will be used only to populate the TableName field.

If the query includes a synonym containing a period, it must be enclosed in escaped quotes. For example: select * from \"synonym.name\"

If using a query when the output of a DatabaseReader source is the input of a DatabaseWriter target, specify the target table name as the value of DatabaseReader's Tables field.

Quiesce on IL Completion

Boolean

False

Restart Behavior on IL Interruption

Enum

Keep target table data

See Fast Snapshot Recovery during initial load.

Return DateTime As

String

Joda

Set to String to return timestamp values as strings rather than Joda timestamps. The primary purpose of this option is to avoid losing precision when microsecond timestamps are converted to Joda milliseconds. The format of the string is yyyy-mm-dd hh:mm:ss.ffffff. When the source is BigQuery or Snowflake, this setting is ignored and timestamp values are always returned as Java DataTime.

SSL Config

String

If the source is Oracle and it uses SSL, specify the required SSL properties (see the notes on SSL Config in Oracle Reader properties).

Tables

String

Specify the table(s) or view(s) to be read.

MySQL, Oracle, and PostgreSQL names are case-sensitive, SQL Server names are not. Specify names as <dataset name>.<table name> for BigQuery, <database name>.<table name> for MySQL, <schema name>.<table name> for Oracle and PostgresQL, <schema name>.<table name> for Snowflake, and <database name>.<schema name>.<table name> for SQL Server.

You may specify multiple tables and views as a list separated by semicolons or with the % wildcard. For example, HR% would read all the tables whose names start with HR. You may use the % wildcard only for tables, not for schemas or databases. The wildcard is allowed only at the end of the string: for example, mydb.prefix% is valid, but mydb.%suffix is not.

If you are using the Query property, specify QUERY as the table name.

Modifying this property can interfere with recovery. If recovery is enabled for the application and it is necessary to modify the tables list, export the application (see Apps page), drop it, update the exported TQL with the new values, and import it (see Apps page).

Use Connection Profile

Boolean

False

See Introducing connection profiles. Appears in web UI only when Database Provider Type is BigQuery or Snowflake.

Username

String

Specify the DBMS user name the adapter will use to log in to the server specified in ConnectionURL.

For all databases, this user must have SELECT permission or privileges on the tables specified in the Tables property. For Oracle, this user must also have SELECT privileges on DBA_TAB_COLS and ALL_COLL_TYPES.

Vendor Configuration

Striing

When reading from an Oracle PDB database, specify the PDB container name as follows, replacing <PDB name> with the name of the container:

{"Database" : { "Oracle" : { "CONTAINER" : "<PDB name>"}}}

Types.ARRAY

java.lang.String

Types.BIGINT

java.lang.Long

Types.BIT

java.lang.Boolean

Types.CHAR

java.lang.String

Types.DATE

org.joda.time.LocalDate

Types.DECIMAL

java.lang.String

Types.DOUBLE

java.lang.Double

Types.FLOAT

java.lang.Double

Types.INTEGER

java.lang.Integer

Types.NUMERIC

java.lang.String

Types.REAL

java.lang.Float

Types.SMALLINT

java.lang.Short

Types.TIMESTAMP

org.joda.time.DateTime

Types.TINYINT

java.lang.Short

For MySQL, if the source tables contain columns of this type, append ?tinyInt1isBit=false to the connection URL (jdbc:mysql://<ip address>:<port>/<database name>?tinyInt1isBit=false).

Types.VARCHARCHAR

java.lang.String

other types

java.lang.String

ARRAY

java.lang.String

BIGNUMERIC

java.math.BigDecimal

BOOL

java.lang.Boolean

BYTES

java.lang.String

DATE

java.time.LocalDate

DATETIME

java.time.LocalDateTime

FLOAT64

java.lang.Double

GEOGRAPHY

java.lang.String

INT64

java.lang.Long

INTERVAL

java.lang.String

JSON

java.lang.String

NUMERIC

java.math.BigDecimal

RANGE

java.lang.String

STRING

java.lang.String

STRUCT

java.lang.String

TIME

java.lang.String

TIMESTAMP

java.time.ZonedDateTime