Skip to main content

Striim Platform 5.0 documentation

Fabric Mirror Writer programmer's reference

Fabric Mirror Writer properties

property

type

default value

notes

Batch Policy

String

eventcount:10000, interval:5m

You may specify eventcount and/or interval, or alternatively filesize (which cannot be combined with eventcount or interval). With the default policy of eventcount:10000,interval:5m events will be written to the Fabric landing zone every five minutes or sooner if Striim accumulates 10,000 events. The filesize value range is 10MB minimum to 2048M maximum; do not set it outside this range. For best performance with a large dataset, set only the interval. See Setting output names and rollover / upload policies for more information.

Connection Profile Name

String

The connection profile for the Microsoft Fabric instance where data will be mirrored. See Introducing connection profiles.

Connection Retry Policy

String

retryInterval=30, maxRetries=3

Specifies the number of times and the time interval between subsequent retries the writer tries to upload a file on a connection drop.

Mirrored Database Name

String

The name of the database in which source data will be mirrored. If the database does not exist, Striim will create it.

Parallel Threads

Integer

For initial load using Database Reader, creates multiple instances of the writer for better performance. When using the wizard, by default this is set to 4 in the initial load app. For maximum performance, set to one less than the number of virtual CPU cores in the SQL2Fabric-Mirroring service.

In the CDC phase, leave blank or set to 1. Parallel threads are not supported in this phase as they can interfere with recovery.

For more information, see Creating multiple writer instances (parallel threads).

Tables

String

Specify the names of the tables to mirror in the format <source>,<target>. Fabric names may not contain special characters.

You can provide multiple mappings with a semicolon as a separator. For example: srcdb.emp1,schema.emp1; srcdb.emp2,schema.emp2. You can also provide wildcards to specify table names, for example: <source schema>.%,<target schema>.% If a table name contains a special character, it will be converted to a valid Java identifier in Fabric; for example, TestDB$Table will be converted to TestDB_DOLLAR_Table.

Optionally, specify KeyColumns with table mapping using the syntax <source table>,<target table> KeyColumns(<target column>). Null values for key columns are not supported.

When the source is SQL Server, during initial load using Database Reader, source table names must be specified in three parts (<source database>.<source schema>.<source table>,<target schema>.<target table>), but in the CDC phase, when the source is MS SQL Reader or MS Jet, source table names must be specified in two parts (<source schema>.<source table>,<target schema>.<target table>). The schemas must exist in the target. Table names are case-sensitive.

Fabric Mirror Writer Sample TQL

The following would mirror SQL Server data to Fabric.

CREATE SOURCE SQLServerCDCIn USING MSSqlReader (
  Username:'wauser',
  Password:'password',
  DatabaseName:'mysourcedb',
  ConnectionURL:'192.168.1.10:1433',
  Tables:'dbo.%'
) 
OUTPUT TO SQLServerCDCStream;

CREATE TARGET FabricMirrorTarget USING FabricMirrorWriter ( 
  MirroredDatabaseName: 'mymirrordb', 
  ConnectionProfileName: 'FabricCP', 
  Tables: 'dbo.%,db.%'
) 
INPUT FROM SQLServerCDCStream;
Fabric Mirror Writer data type support and mapping

Data types not listed are not supported.

MariaDB data type

Fabric data type

BIGINT

bigint

BINARY(16)

varbinary

BIT(8)

varbinary

BLOB

varbinary

BOOLEAN

int

CHAR(5)

varchar

DATE

date

DATETIME

datetime2

DECIMAL

decimal

DOUBLE

float

FLOAT

float

INT

int

INT UNSIGNED

int

JSON

varchar

LONGBLOB

varbinary

LONGTEXT

varchar

MEDIUM BLOB

varbinary

MEDIUM INT

int

MEDIUM TEXT

varchar

SERIAL

bigint

SMALL INT

int

TEXT

varchar

TIMESTAMP

datetime2

TINYBLOB

varbinary

TINYINT

int

TME

time

VARBINARy(255)

varbinary

VARCHAR(100)

varchar

YEAR

int

MariaDB Xpand data type

Fabric data type

BIGINT

bigint

BINARY(16)

varbinary

BOOLEAN

int

CHAR(10)

varchar

DATE

date

DATETIME

datetime

DECIMAL(15,5)

decimal

DOUBLE

float

FLOAT

float

INT

Bigint

JSON

varchar

MEDIUMINT

int

TEXT

varchar

TIMESTAMP

datetime2

TINYINT

int

VARBINARY(255)

varbinary

VARCHAR(255)

varchar

MySQL data type

Fabric data type

BIGINT

bigint

BIT(1)

varbinary

BLOB

varbinary

BOOLEAN DEFAULT TRUE

int

CHAR(36) DEFAULT (UUID())

varchar

CHAR(50)

varchar

DATE

date

DATETIME

datetime2

DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP

datetime2

DECIMAL(10, 2)

decimal

DOUBLE

float

FLOAT

float

INT

int

INT UNSIGNED

int

JSON

varchar

LONGBLOB

varbinary

MEDIUMINT

int

SMALLINT

int

TEXT

varchr

TIME

datetime2

TIMESTAMP DEFAULT CURRENT_TIMESTAMP

datetime2

TINYINT

int

VARBINARY(255)

varbinary

VARCHAR(100)

varchar

YEAR

int

Oracle data type

Fabric data type

BINARY_DOUBLE

varbinary

BINARY_FLOAT

real

BLOB

varbinary

CHAR(10)

varchar

CLOB,

varchar

DATE

date

FLOAT(10)

real

INTEGER

decimal

INTERVAL DAY TO SECOND

varchar

INTERVAL YEAR TO MONTH

varchar

NCHAR(10)

varchar

NCLOB

varchar

NUMBER(10)

decimal

NVARCHAR2(50)

varchar

RAW(2000)

varbinary

TIMESTAMP(6)

datetime2

TIMESTAMP(6) WITH TIME ZONE

varchar

TIMESTAMP(6) WITH LOCAL TIME ZONE

bigint

VARCHAR2(50)

varchar

PostgreSQL data type

Fabric data type

bigint

bigint

boolean

bit

bytea

varbinary

char(5)

varchar

circle

varchar

date

date

decimal (10,2)

decimal

double precision

float

inet

varchar

int4range

varchar

integer

int

integer check(unsigned number >=0)

int

interval

varchar

json

varchar

jsonb

varchar

macaddr

varchar

money

varchar

point

varchar

real

real

serial

int

serial

int

smallint

int

smallint

int

text

varchar

time

datetime2

timestamp

datetime2

timestamptz

varchar

tsvector

varchar

varchar(100)

varchar

varchar(100) collate "c"

varchar

varchar(255)

varchar

xml

varchar

SQL Server data type

Fabric data type

bigint

bigint

binary(5)

varbinary

bit

bit

char(10)

varchar

date

date

datetime

datetime2

datetime2(7)

datetime2

datetimeoffset

datetime2

decimal(10,2)

decimal

decimal(38,2)

decimal

float

float

int

int

int identity(10,2)

int

nchar(10)

varchar

ntext

varchar

numeric(10,2)

decimal

numeric(38,2)

decimal

nvarchar(50)

varchar

nvarchar(max)

varchar

real

float

real(10)

float

smalldatetime

datetime2

smallint

int

text

varchar

time(7)

datetime2

timestamp

datetime2

tinyint

int

uniqueidentifier

varchar

varbinary(50)

varbinary

varchar(50)

varchar

xml

varchar

In this section: