Snowflake initial load

You can use Database Reader for initial load to create a point-in-time copy of your existing source Snowflake dataset at the target. This establishes a baseline for subsequent continuous updates.

You can leverage Database Reader's parallel processing capabilities to accelerate data loading for large datasets, while the system automatically handles schema creation and data type mapping between Snowflake and target systems. You can configure table selection with wildcard support, create custom SQL queries for selective data extraction, and optimize batch sizes for performance tuning..

Snowflake setup for initial load

In Snowflake, create a user for use by Striim with SELECT privileges on the tables to be read.

Database Reader properties for Snowflake sources

property	type	default value	notes
Connection Profile Name	enum		Appears in Flow Designer only when Use Connection Profile is True. See Connection profiles.
Connection URL	String		When connecting through an SSH tunnel (see Using an SSH tunnel to connect to a source or target), specify the IP address of the tunnel. Specify as `jdbc:snowflake://<account identifier>.snowflakecomputing.com?db=<database name>` or use a connection profile (see Connection profiles)
Create Schema	Boolean	False	If set to True, when Database Reader's output stream is the input stream of an Azure Synapse Writer, BigQuery Writer, Database Writer (for MariaDB, MariaDB Xpand, MySQL, Oracle, PostgreSQL, SQL Server, or YugabyteDB only), Databricks Writer, Fabric Data Warehouse Writer, Fabric Mirror Writer, Iceberg Writer, Snowflake Writer, or Spanner Writer target, the schema(s) and tables specified in the Tables property in the writer will be created in the target. For example, if the Tables property in the writer is `sourceschema1.%,targetschema1.%;sourceschema2.%,targetschema2.%` then the schemas targetschema1 and targetschema2 will be created in the target, and all the tables in sourceschema1 and sourceschema2 will be created in the corresponding target schema. Data types will be mapped as per Target data type support & mapping for Snowflake sources. Single-column PRIMARY KEY constraints will be replicated in the target. DEFAULT, FOREIGN KEY, NOT NULL, composite PRIMARY KEY, and UNIQUE KEY constraints will not be replicated. If a table already exists in the target, Striim will verify that the source and target structure match. If they do not, the application will halt and log a TargetTableSchemaMismatchException. MON output for the target tables will include a schemaCreationStatus of Pending, InProgress, Success, or Failed.
Database Provider Type	String	Default	Set to Snowflake. Controls which icon appears in the Flow Designer and whether Use Connection Profile appears in the web UI.
Excluded Tables	String		Data for any tables specified here will not be returned. For example, if `Tables` uses a wildcard, data from any tables specified here will be omitted. Multiple table names (separated by semicolons) and wildcards may be used exactly as for `Tables`.
Fetch Size	Integer	100	Sets the maximum number of records to be fetched from the database in a single JDBC method execution (see the discussion of fetchsize in the documentation for your JDBC driver).
JAAS Configuration	String		Not applicable to Snowflake in this release.
Password	encrypted password		The password for the specified user. See Encrypted passwords.
Parallel Threads	Integer	1	For Fast Snapshot Loading to Azure Synapse, BiqQuery, Databricks, Fabric Data Warehouse, Microsoft Dataverse, or Snowflake, specify the number of threads Database Reader will use. This value should not be higher than the number of tables to be read. For other targets, leave set to the default of 1. When Database Reader is started, the tables will be distributed among the threads in round-robin fashion, then each thread will read one table at a time. When the Tables property is a list of tables, the tables are read in that order. If the Tables property uses a wildcard, the sequence is determined by the order in which the JDBC driver gives Database Reader the list of tables. Known issue DEV-49013: the maximum number of tables supported by Parallel Threads is 255.
Query	String		Optionally, specify a single SQL SELECT statement specifying the data to return. You may query tables, aliases, synonyms, and views. Query is not supported when Create Schema is True, Parallel Threads is greater than 1, or Restart Behavior on IL Interruption is Truncate_target_table or Replace_target_table. When `Query` is specified and `Tables` is not, the WAEvent TableName metadata field value will be QUERY. When both `Query` and `Tables` are specified, the data specified by `Query` will be returned, and the `Tables` setting will be used only to populate the TableName field. If the query includes a synonym containing a period, it must be enclosed in escaped quotes. For example: `select * from \"synonym.name\"` If using a query when the output of a DatabaseReader source is the input of a DatabaseWriter target, specify the target table name as the value of DatabaseReader's Tables field.
Quiesce on IL Completion	Boolean	False	With the default value of False, you must stop the application manually after all data has been read. Set to True to automatically quiesce the application after all data has been read (see discussion of QUIESCE in Console commands). When you see on the Apps page that the application is in the Quiescing state, it means that all the data that existed when the query was submitted has been read and that the target(s) are writing it. When you see that the application is in the Quiesced state, you know that all the data has been written to the target(s). At that point, you can undeploy the initial load application and then start another application for continuous replication of new data.Console commands Note Set to True only if all targets in the application support auto-quiesce (see Writers overview).Writers overview
Restart Behavior on IL Interruption	Enum	Keep target table data	See Fast Snapshot Recovery during initial load.
Return DateTime As	String	Joda	For Snowflake, this setting is ignored and timestamp values are always returned as Java DateTime.
SSL Config	String		Not applicable to Snowflake in this release.
Tables	String		Specify the table(s) or view(s) to be read. Specify names as `<schema name>.<table name>`. Names are case-sensitive. You may specify multiple tables and views as a list separated by semicolons or with the `%` wildcard. For example, `HR%` would read all the tables whose names start with HR. You may use the `%` wildcard only for tables, not for schemas or databases. The wildcard is allowed only at the end of the string: for example, `mydb.prefix%` is valid, but `mydb.%suffix` is not. If you are using the Query property, specify QUERY as the table name. Modifying this property can interfere with recovery. If recovery is enabled for the application and it is necessary to modify the tables list, export the application (see Apps page), drop it, update the exported TQL with the new values, and import it (see Apps page).
Use Connection Profile	Boolean	False	See Connection profiles.
Username	String		Specify the user name the adapter will use to log in to the server specified in ConnectionURL. This user must have SELECT permission or privileges on the tables specified in the Tables property.
Vendor Configuration	Striing		Not applicable to Snowflake in this release.

Sample Database Reader WAEvent

For the following row:

id  first_name  last_name  phone  street          city          state  zip_code
1   Deborah     Burks      NULL   9273 Thorne AV  Orchard Park  NY     14127

The WAEvent would be similar to:

WAEvent{
  data: [1,"Deborah","Burks",null,"9273 Thorne AV","Orchard Park","NY","14127"]
  metadata: {"TableName":"BikeStores.sales.customers","ColumnCount":8,
    "OperationName":"SELECT","OPERATION_TS":1681412863364}
  userdata: null
  before: null
  dataPresenceBitMap: "fwM="
  beforePresenceBitMap: "AAA="
  typeUUID: {"uuidstring":"01edda2e-77f7-9b21-83c2-8e859085da65"}
};

The operation name for Database Reader WAEvents is always SELECT.

DatabaseReader data type support and correspondence for Snowflake sources

Snowflake type	Aliases	TQL type
ARRAY	-	java.lang.String
BINARY	VARBINARY	java.lang.String
DATE	-	java.time.LocalDate
FLOAT	DOUBLE, DOUBLE PRECISION, FLOAT4, FLOAT8, REAL	java.lang.Double
GEOGRAPHY	-	java.lang.String
GEOMETRY	-	java.lang.String
NUMBER	BIGINT, BYTEINT, DECIMAL, INT, INTEGER, NUMERIC, SMALLINT, TINYIN	java.math.BigDecimal
OBJECT	-	java.lang.String
TIME	-	java.time.String
TIMESTAMP_LTZ	TIMESTAMP WITH LOCAL TIME ZONE, TIMESTAMPLTZ	java.time.ZonedDateTime
TIMESTAMP_NTZ	DATETIME, TIMESTAMP, TIMESTAMP WITHOUT TIME ZONE, TIMESTAMPNTZ	java.time.LocalDateTime
TIMESTAMP_TZ	TIMESTAMP WITH TIME ZONE, TIMESTAMPTZ	java.time.ZonedDateTime
VARCHAR	CHAR VARYING, NCHAR VARYING, NVARCHAR, NVARCHAR2, STRING, TEXT CHAR, CHARACTER, NCHAR	java.lang.String
VARIANT	-	java.lang.String

Snowflake initial load

Snowflake setup for initial load

Database Reader properties for Snowflake sources

Note

Sample Database Reader WAEvent

DatabaseReader data type support and correspondence for Snowflake sources

Search results