Skip to main content

Striim for Databricks Documentation

Configure Azure Databricks

Before you can create a Striim for Databricks pipeline, you must do the following:

  1. Create an Azure Databricks user that Striim for Databricks will use to connect to the target (see Learn / Manage users / Add users to your Azure Databricks account).

  2. Give that user the following privileges

    1. CREATE SCHEMA

    2. if you are using Unity Catalog, also USE CATALOG (not required if you are using the Hive metastore)

  3. Get a personal access token for that user, which you will need to provide when creating a pipeline. The following steps are subject to change by Microsoft but they worked in April 2023:

    1. In Azure Databricks, at the top right corner, click your login name, select User Settings, and click Generate new token.

    2. In the Comment field enter a description for the token (such as for Striim), optionally adjust its Lifetime, and click Generate.

    3. Copy the token and keep it in a safe place. If you lose it you will need to create a new token.

  4. Get the JDBC URL for your Databricks cluster, which you will need to provide when creating a pipeline. The following steps are subject to change by Microsoft but they worked in April 2023:

    1. In Azure Databricks, at the top of the left navigation panel, select Data Science & Engineering.

      DataScience.png

    2. In the left navigation panel, click Compute, then click the name of your cluster.

    3. At the bottom of the page, expand Advanced options, then select the jDBC/ODBC tap.

    4. Copy the JDBC URL (it starts with jdbc and ends with PWD=<personal-access-token>).

In this section: