Fabric Data Warehouse Writer programmer's reference

Azure Account Access Key

encrypted password

the account access key for the storage account you created as described in Initial Fabric Data Warehouse Writer setup, which you can find in Azure at Storage accounts > <account name> > Access keys

Azure Account Name

String

the name of the storage account you created as described in Initial Fabric Data Warehouse Writer setup, which you can find in Azure at Storage accounts

CDDL Action

String

Process

See Handling schema evolution.Handling schema evolution

If TRUNCATE commands may be entered in the source and you do not want to delete events in the target, precede the writer with a CQ with the select statement SELECT * FROM <input stream name> WHERE META(x, OperationName).toString() != 'Truncate'; (replacing <input stream name> with the name of the writer's input stream). Note that there will be no record in the target that the affected events were deleted.

Known issue DEV-48379: ALTER TABLE ... ADD PRIMARY KEY is not supported by Fabric, but instead of halting the application will continue as if the command had been successful.

Client Configuration

String

If using a proxy, specify ProxyHost=<host name or IP address>,ProxyPort=<port number>.

Column Delimiter

String

If the data to be written may contain the default column delimiter (ASCII / UTF-8 124), specify a different delimiter that will never appear in the data.

Connection Profile Name

String

Appears in Flow Designer only when Use Connection Profile is True. See Connection profiles.

Connection Retry Policy

String

initialRetryDelay=10s, retryDelayMultiplier=2, maxRetryDelay=1m, maxAttempts=10, totalTimeout=10m

With the default setting, if a connection attempt is unsuccessful, the adapter will try again in 10 seconds (InitialRetryDelay=10s). If the second attempt is unsuccessful, in 20 seconds it will try a third time (InitialRetryDelay=10s multiplied by retryDelayMultiplier=2). If that fails, the adapter will try again in 40 seconds (the previous retry interval 20s multiplied by 2). If connection attempts continue to fail, the the adapter will try again every 60 seconds (maxRetryDelay=1m) until a total of 10 connection attempts have been made (maxAttempts=10), after which the adapter will halt and log an exception.

The adapter will halt when either maxAttempts or totalTimeout is reached.

InitialRetryDelay,maxRetryDelay, and totalTimeout may be specified in milliseconds (ms), seconds (s, the default), or minutes (m).

If retryDelayMultiplier is set to 1, connection will be attempted on the fixed interval set by InitialRetryDelay.

To disable connection retry, set maxAttempts=0.

Negative values are not supported.

Connection URL

String

Provide the full JDBC connection URL with Active Directory password for authentication using the format:

jdbc:sqlserver://serverName=<host URL>;encrypt=strict;
Authentication=<Active Directory password>;
database=<data warehouse name>

Excluded Tables

String

If Tables uses a wildcard, data from any tables specified here will be omitted. Multiple table names (separated by semicolons) and wildcards may be used exactly as for Tables.

External Stage Connection Profile Name

String

Appears in Flow Designer only when Use Connection Profile is True. See Connection profiles.

Ignorable Exception Code

String

Set to TABLE_NOT_FOUND to prevent the application from terminating when Striim tries to write to a table that does not exist in the target. See Handling "table not found" errors for more information.

Ignored exceptions will be written to the application's exception store (see CREATE EXCEPTIONSTORE).

Mode

String

MERGE

With the default value of MERGE, inserts and deletes in the source are handled as inserts and deletes in the target. With this setting:

For more information about keycolumns, see Defining relations between source and target using ColumnMap and KeyColumns.

Set to APPENDONLY to handle all operations as inserts. With this setting:

Optimized Merge

Boolean

False

Appears in Flow Designer only when Mode is MERGE.

Set to True only when Mode is MERGE and the target’s input stream is the output of an HP NonStop reader, MySQL Reader, or Oracle Reader source and the source events will include partial records.

For example, with Oracle Reader, when supplemental logging has not been enabled for all columns, partial records are sent for updates. When the source events will always include full records, leave this set to false.

Set to True also when the source is Oracle Reader and the source table includes BLOB or CLOB columns

When Optimized Merge is True, each primary key update is handled in a separate write operation. If the source has frequent primary key updates, this may lead to a decline in write performance compared with Optimized Merge = False.

Parallel Threads

String

Creates multiple instances of writer for better performance. See Creating multiple writer instances.Creating multiple writer instances

Not supported when Mode is MERGE.

Password

encrypted password

Microsoft Fabric password.

Tables

String

The name(s) of the table(s) to write to, in the format <source database>.<source table>,<target warehouse>.<target schema>.<target table>. All tables must be in the same warehouse.

Table names are case-sensitive.

You can provide multiple mappings with a semicolon as a separator. For example: srcdb.emp1,warehouse.schema.emp1; srcdb.emp2,warehouse.schema.emp2

You can also provide wildcards to specify table names. For example: <sourcedb>.%,<warehouse>.<targetSchema>.%

Optionally, specify KeyColumns and ColumnMap along with table mapping For example: <source table>,<target table> KeyColumns(<target column>) ColumnMap(<target column>=<source column>,...)

Upload Policy

String

eventcount:10000, interval:5m

The upload policy may include eventcount and/or interval (see Setting output names and rollover / upload policies for syntax).Setting output names and rollover / upload policies

Events are buffered locally on the Striim server and sent as a batch to the target every time either of the specified values is exceeded.

Trusted Workspace Access

Boolean

False

Set to True to enable trusted workspace access for secure connectivity between Fabric workspaces and private Azure Storage accounts. When enabled, Fabric Data Warehouse Writer uses Fabric workspace identity to access the private storage account through trusted workspace access instead of public network access. Requires prior configuration of workspace identity, storage account role assignments, trusted workspace access, and private endpoints. See Using trusted workspace access with Fabric Data Warehouse Writer for setup instructions.

Use Connection Profile

Boolean

See Connection profiles.

Username

String

Microsoft Fabric username.