Fabric Data Warehouse Writer
Fabric Data Warehouse Writer writes to tables in a data warehouse in Microsoft Fabric.
Summary
APIs used/data supported | Uses the SQL endpoint of the Microsoft Fabric Data Warehouse to write data to tables. |
Supported formatters | Formatters are not supported with this writer. |
Supported sources | All sources supported by Striim that generate the following event types:WAEvent and typed events. See Readers overview. |
Security and authentication | Supports authentication using Active Directory Password with Fabric username and password. Authentication using OAuth is not supported in this release. |
Resilience / recovery |
|
Performance | Supports parallel threads for improved performance in some circumstances. See Creating multiple writer instances (parallel threads). |
Programmability |
|
Metrics and auditing | Key metrics available through Striim monitoring. See Fabric Data Warehouse Writer monitoring metrics. |
Typical use case and integration
Data engineers can stream data into a Fabric Data warehouse and provide access to business teams to generate Power BI dashboards for insights.
Fabric Data Warehouse Writer overview
Microsoft Fabric is an all-in-one analytics platform created for businesses and data professionals. The platform handles everything from data science and real-time analytics to data storage and data migration.
All of the data utilized within Fabric is stored in OneLake. OneLake is a single, unified, logical data lake that’s responsible for supporting all of the Fabric workloads. OneLake is built on top of Azure Data Lake Storage (ADLS) Gen2 and can support any type of file, structured or unstructured. All Fabric data items like data warehouses and lakehouses store their data automatically in OneLake in Delta Parquet Table format. On the other hand, raw files (unstructured) can be stored under the "Files" folder in the OneLake storage (see Fabric Lakehouse File Writer).
Fabric Data Warehouse Writer is a target adapter that writes to the SQL endpoint of the Microsoft Fabric Data Warehouse tables. It uses ADLSGen2 as the temporary staging location before copying the data to the Fabric Data Warehouse tables. Fabric Data Warehouse Writer authenticates the staging location using the Storage Account’s name and its Access Key. You will need to provision and have ADLSGen2 access. Fabric Data Warehouse Writer supports authentication using Active Directory Password with a Fabric username and password. Authentication using OAuth is not currently supported.
Fabric Data Warehouse Writer data type support and correspondence
Fabric Data Warehouse Writer supports the TQL data types and a Fabric Datawarehouse type table.
The following is the mapping between TQL types and Fabric types.
TQL type | Fabric type |
|---|---|
Byte | smallint |
Double, Float | float |
Integer | int |
Long | bigint |
Short | smallint |
String | char, varchar |
Date | date |
DateTime | datetime2 |
Time | time |
Boolean | bit |
Binary | varbinary |
Decimal | decimal |