Skip to main content

Striim for Databricks Documentation

Connect to PostgreSQL

When prompted by the wizard, enter the appropriate connection details.

  • Where is the database located? If your source is an Amazon Aurora for PostgreSQL, Amazon RDS for PostgreSQL , Azure Database for PostgreSQL, or Cloud SQL for PostgreSQL instance, select that, otherwise leave set to the default.

  • Hostname: Enter the IP address or fully qualified network name of the instance (for example, 198.51.100.10 or mydb.123456789012.us-east-1.rds.amazonaws.com) or, if you are connecting via an SSH tunnel, paste the string copied from Striim Cloud Console > Service details > Secure connection > Tunnel Address (see Configure Striim to use your SSH tunnel).

  • Port: Enter the port for the specified host.

  • Username: Enter the name of the user you created when you Set up your PostgreSQL source.

  • Password: Enter the password associated with the specified user name.

  • Database name: Enter the name of the source database.

  • Connect using SSL: Select if connecting to the source database using SSL. See the detailed instructions below.

  • Name your source connection: Enter a descriptive name, such as PostgreSQLConnection1.

  • How would you like to handle schema changes?: choose the option appropriate for your workflow.

    • Do not propagate changes and continue (default): Striim will take no action. Any data added to new tables will not be synced to Databricks. Any data added to a new column will not be synced to Databricks as the column will not exist in the target. Tables dropped from the source will continue to exist in Databricks.

    • Pause the pipeline: Striim will pause the pipeline. After making any necessary changes in the source or Databricks, restart the pipeline.

    • Propagate changes to Databricks: In Databricks, Striim will create a new table, add a column, or drop a table so that the target matches the source. Sync will continue without interruption. (Note that if a column is dropped from a source table, it will not be dropped from the corresponding Databricks target table.)

  • What is your PostgreSQL replication slot: Enter the name of the slot you created or chose in Set up your PostgreSQL source . Note that you cannot use the same slot in two pipelines, each must have its own slot.

Use SSL with on-premise PostgreSQL

  1. Get an SSL certificate in .pem format from your database administrator (see Creating Certificates in the PostgreSQL documentation).

  2. Convert that to .pk8 format (replace <file name> with the name of the .pem file):

    openssl pkcs8 -topk8 -inform PEM -outform DER -in <file name>.pem -out client.root.pk8 \
  -nocrypt

  3. Set these properties in Striim:

    • SSL Mode: enter disable , allow , prefer , require , or verify-ca to match the type of encryption and validation required for the user (verify-full is not supported).

    • SSL Certificate: upload the client-cert.pem file created in step 2

    • SSL Certificate Key: upload the client.root.pk8 file created in step 2

    • SSL Root Certificate: upload the server-ca.pem file created in step 2

Use SSL with Amazon Aurora or RDS for PostgreSQL

  1. Download the root certificate rds-ca-2019-root.pem (see AWS > Documentation > Amazon Relational Database Service (RDS) Using SSL with a PostgreSQL DB instance).

  2. Set these properties in Striim:

    • SSL Mode: enter disable , allow , prefer , require , or verify-ca to match the type of encryption and validation required for the user (verify-full is not supported).

    • SSL Root Certificate: upload the file you downloaded in step 1

Use SSL with Azure Database for PostgreSQL

  1. Download the root certificate DigiCertGlobalRootG2.crt.pem (see Learn > Azure > PostgreSQL > Configure TLS connectivity in Azure Database for PostgreSQL - Single Server).

  2. Set these properties in Striim:

    • SSL Mode: enter disable , allow , prefer , require , or verify-ca to match the type of encryption and validation required for the user (verify-full is not supported).

    • SSL Root Certificate: upload the file you downloaded in step 1

Use SSL with Cloud SQL for PostgreSQL

  1. Download server-ca.pem, client-cert.pem & client-key.pem from Google Cloud Platform (see Cloud SQL > Documentation > PostgreSQL > Guides > Configure SSL/TLS certificates).

  2. Convert client-key.pem to .pk8 format:

    openssl pkcs8 -topk8 -inform PEM -outform DER -in client-key.pem \
  -out client.root.pk8 -nocrypt

  3. Set these properties in Striim:

    • SSL Mode: enter disable , allow , prefer , require , or verify-ca to match the type of encryption and validation required for the user (verify-full is not supported).

    • SSL Certificate: upload the client-cert.pem file you downloaded in step 1

    • SSL Certificate Key: upload the client.root.pk8 file you created in step 2

    • SSL Root Certificate: upload the server-ca.pem file you downloaded in step 1

In this section: