Skip to main content

Connection profiles

A connection profile defines the authentication and connection attributes required to access an external system from within the Striim platform. These attributes may include, among other things:

  • Usernames and passwords

  • OAuth tokens

  • API keys

  • Access key/secret key pairs

  • Entra ID (formerly Azure AD) tokens

All sensitive fields, such as passwords and tokens, are securely encrypted (AES-256b) within Striim’s metadata repository. They are never displayed in clear text after being saved, even to privileged users.

Once defined, a connection profile can be reused across multiple sources, targets, and other components in multiple applications to connect to the same external resource. For example, you can define a connection profile to connect to Salesforce by authenticating using OAuth. You can then use the connection profile with Salesforce Reader, Salesforce CDC Reader, or Salesforce Writer; you do not have to enter the connection credentials every time you build an application that connects to Salesforce.

Connection profiles reduce the overhead of maintaining connectivity to external resources. As a Striim developer, you can build and maintain multiple data pipelines that move data between business-critical systems, such as Salesforce, Snowflake, or BigQuery. You can use connection profiles to connect to these systems and remain connected with minimum effort. For example, if you modify or update any of these attributes such as username, password, or personal access token, or if any of these attributes such as OAuth are automatically refreshed, then these changes will automatically cascade to every data pipeline where the connection profile is used, thus ensuring that your pipelines always remain connected to the external systems.

Connection profiles also enable you to control access to the connection credentials required to connect to an external system. You can use the fine-grained permissions on the connection profile object to regulate access to the connection profile. For example, as a Striim Admin, you can provide edit access to the set of users designated to manage access credentials for your company, while granting read-only access to other users.

Managing connection profiles

Permissions associated with connection profiles

Before using a connection profile, you must assign the appropriate Striim permissions to the users.

Permission

What the user can do

READ + SELECT

  • Search for the connection profile

  • Use the connection profile in applications

  • View the connection profile details (not including sensitive fields such as passwords)

CREATE

  • Create connection profiles

EDIT

  • Edit the connection profile

DELETE

  • Delete the connection profile

As an example, a Striim admin could enable the role dataengr in the namespace SharedCPs to search for and use all connection profiles in that namespace as follows:

GRANT READ, SELECT ON connectionprofile SharedCPs TO ROLE SharedCPs.dataengr;

Alternatively, a Striim admin could enable the role oradataengr in the namespace OracleDBNS to search for and use the single connection profile OracleCP in that same namespace as follows:

GRANT READ, SELECT ON connectionprofile OracleDBNS.OracleCP TO ROLE OracleDBNS.oradataengr;

See “Managing permissions and roles” for more information on the commands to use to grant permissions to users.

Connection profile lifecycle

The lifecycle of a connection profile encompasses every stage from creation, configuration, testing, validation, active use in applications, and updates to eventual deletion.

Creating a connection profile

Note

Note: Striim is introducing connection profiles in phases over upcoming releases. Have a component or authentication method you’d like to see supported sooner? Let Striim Support know by submitting an enhancement request.

You can create a connection profile in two ways:

  • from the Manage Striim > Connection Profiles page

  • from a Striim reader or writer component in the Flow Designer

You must have CREATE permissions for connection profiles to be able to create a connection profile.

Creating a connection profile from the Manage Striim > Connection Profiles page

  1. Select Manage Striim > Connection Profiles > Add Connection Profile.

  2. Enter a unique Connection Profile Name to easily identify the external resource.

  3. From the Namespace drop-down, select the namespace where the profile will be stored. All users who will use this profile must have READ and SELECT permissions on the connectionprofile component in that namespace.

  4. From the Endpoint Type dropdown, select the type that matches the external service.

  5. If the Authentication Type drop-down appears, select the authentication type you want to use. You can create multiple connection profiles for the same external service with each profile using a different authentication type, or the same authentication type with different settings.

  6. Enter values for the connection properties as described in the inline help, then click Test to validate the connection.

  7. If the test is successful, click Save, otherwise correct the properties as necessary and test again. You can save the changes to the connection profile without a successful test, but if the connection profile is not valid, applications that use it will halt.

Creating a connection profile from a source or target in the Flow Designer

  1. If the component supports connection profiles, the property panel will include a Use Connection Profile toggle (disabled by default). Turn on this toggle.

  2. A dropdown will be shown with the following options:

    • + New Connection Profile

    • A list of existing connection profiles for which you have READ and SELECT permissions for that endpoint type (for example, BigQuery or Salesforce).

    Click + New Connection Profile. This will open a modal popup with the connection profile attributes.

  3. In Connection Profile Name, enter a name that uniquely identifies the external resource.

  4. From the Namespace drop-down, select the namespace where the profile will be stored. You must have CREATE permission on connectionprofile objects in that namesapce. All users who will use this profile must have READ and SELECT permissions on the connection profile.

  5. From the Endpoint Type dropdown, select the type that matches the external service.

  6. If the Authentication Type drop-down appears, select the authentication type you want to use. You can create multiple connection profiles for the same external service with each profile using a different authentication type, or the same authentication type with different settings.

  7. Enter values for the connection properties as described in the inline help, then click Test to validate the connection.

  8. If the test is successful, click Save, otherwise correct the properties as necessary and test again. You can save without a successful test but if the connection profile is not valid applications that use it will halt.

  9. The newly created connection profile will automatically be selected in the component’s property panel. Continue with any additional steps required to create or edit the component.

Viewing connection profiles

You can view details of the connection profiles (except passwords and other sensitive fields, which will always remain masked) and the associated applications. You must have READ and SELECT permissions on the connection profiles to view the details.

  1. Select Manage Striim > Connection Profiles.

  2. For the connection profile you want to view, click ... in the last column and select View. The details will open with all sensitive fields masked.

  3. Click Associated Apps to see the list of Striim applications that use the connection profile.

Using a connection profile in a Striim application

You must have READ and SELECT permissions on the connection profiles to use them in your application.

  1. In the wizard or Flow Designer, after selecting a supported source or target, turn on the Use Connection Profile toggle.

  2. Choose from your existing connection profiles, or create a new one using the Create a Connection Profile flow. You'll be redirected back with the new profile auto-selected.

  3. Continue with the rest of the application creation process.

Updating a connection profile

Users with EDIT permissions can modify connection profiles. All attributes are fully editable when no applications use the profile, but some attributes are locked for in-use profiles to prevent disrupting applications that are currently running..

  1. Select Manage Striim > Connection Profiles.

  2. Next to the connection profile you want to modify, select ... > Edit.

  3. If a property you want to change is uneditable, undeploy all applications that use the connection profile.

  4. Change the properties as desired, then optionally, click Test to validate the connection. If the test fails, correct the properties as necessary and test again.

  5. Click Save.

Deleting a connection profile

You can delete a connection profile if that profile is not associated with any Striim application, component, or feature and if you have DELETE permissions for that profile or for connectionprofile objects in the profile's namespace.

To delete a connection profile:

  1. Select Manage Striim > Connection Profiles

  2. Next to the connection profile you want to modify, select ... > Delete.

  3. Confirm by typing YES in the popup (this action is irreversible). If any deployed applications use the profile, deletion will fail and an error message will appear.

Transitioning to connection profiles

Continuing with current source and target configurations without using connection profiles

You can continue configuring sources and targets without connection profiles. Note that authentication methods may differ between existing adapter configurations and connection profiles. Striim recommends using connection profiles where available for enhanced credential security and reduced application maintenance overhead.

Switching to connection profiles

For existing applications, you can switch to using connection profiles if they are available for the sources and targets in your application.

To modify an existing application to use connection profiles:

  1. Stop and undeploy the application.

  2. Make the changes to the source(s) and/or target(s), as applicable.

  3. Deploy and run the application.

Supported endpoint types and authentication methods for connection profiles

Authentication methods supported in connection profiles

Connection profiles support several authentication methods, grouped into the following categories. When a deployed application uses a connection profile, some attributes are locked while others are editable to avoid unintended disruption to the application. We describe the general behavior for each category when the connection profiles are edited.

  • Authentication using username & password:

    This is the simplest way to connect to external endpoints or systems: by storing the username and password directly in the connection profile. For example, the connection profile for Snowflake supports authentication using username and password.

    Behavior when edited: For some endpoints, editing the password triggers a one-time automatic background reconnection, after which associated apps can use the updated password. Applications also fetch the latest credentials whenever they restart.

  • Authentication using app tokens, key pairs, or service keys:

    In this method, users fetch the tokens provided by the external endpoint or system, typically from the endpoint’s management consoles, and store them in the connection profile. This method usually relies on tokens provided by the external endpoints for programmatic access to data. Examples of endpoints supporting this authentication method include Snowflake, S3 (as staging area for Snowflake or Databricks), Google BigQuery, Stripe, and Zendesk among others.

    Behavior when edited: When the tokens or keys are updated, the associated applications will continue running. On reconnection, they will automatically use the updated credentials. On restart, all applications fetch the latest tokens or keys.

  • Authentication using OAuth 2.0:

    This method uses OAuth 2.0 to allow users to provide Striim the required permissions to access the external endpoint or system. OAuth is a widely used standard for API authorization, enabling secure delegated access. The latest version is OAuth 2.0, which grants access tokens to clients through a multi-step process with the OAuth server.

    • OAuth: Users can authenticate with the external endpoint and grant Striim the required permissions. Striim stores the OAuth tokens—not the login credentials—in the connection profile. Associated applications can use these tokens for access to the external endpoint. When creating or editing the connection profile, Striim must have access to the public internet to be able to access the external resource and complete the OAuth process and, later, to refresh the tokens in the background. Examples of endpoints that support this method of OAuth authentication include Azure Data Lake Storage, Snowflake, and Hubspot.

    • Manual OAuth: If you do not wish to allow Striim to connect to the public internet to get OAuth tokens, you can use "manual OAuth" instead. With this approach, you will manually generate a token using a tool such as CURL or Postman to access the OAuth API, then provide the token to the connection profile. The associated applications can then pick up the token to access the external endpoint. Examples of endpoints that support this approach include Snowflake and Databricks.

    Behavior when edited: When OAuth access or refresh tokens are updated, any associated running applications automatically reconnect in the background using the new tokens—no manual intervention or restart required. If an access token needs to be refreshed (based on the Token Expiry time returned by the OAuth provider), the connection profile automatically refreshes it without affecting consuming applications, minimizing any disruption. If a refresh token is revoked (for example, as part of a security-related token flush), a Striim admin can update the refresh token in the connection profile (see Troubleshooting connection profiles). Once updated, applications automatically use the latest token. As always, applications retrieve the latest tokens whenever they restart.

Supported connection profile endpoints

In this release, Striim supports connection profiles for the following external endpoints or systems.

Endpoint

or External System

Supported Authentication Methods

Notes

Azure Data Lake Storage Gen2 (ADLS-Gen2)

  • Access Key

  • Entra ID (OAuth), formerly known as Azure Active Directory

  • SAS token

connection profiles supported only for using ADLS as the staging area for Databricks and Snowflake

Amazon S3

Access Key

connection profiles supported only for using S3 as the staging area for Databricks and Snowflake

Databricks

  • Entra ID (OAuth) for Azure Databricks

  • Personal Access Token (for Databricks on AWS)

for use with Databricks Writer

Google BigQuery

Service Account Key

for use with BigQuery Writer

Google KMS

Service Account Key

for use with Shield

HubSpot

  • OAuth

  • Private App Token

for use with HubSpot Reader

Microsoft Dynamics 365

OAuth

for use with Microsoft Dynamics 365 Reader and Writer and Microsoft Dynamics 365 Business Central Reader and Writer

Microsoft Fabric Data Warehouse

  • EntraID

  • Entra Service Principal

for use with Fabric Data Warehouse Writer

Microsoft Fabric Lakehouse

  • EntraID

  • Entra Service Principal

for use with Fabric Lakehouse File Writer

Microsoft Fabric mirrored databases

  • Entra Service Principal

  • Entra Password

for use with Fabric Mirror Writer

Salesforce

OAuth

for use with Salesforce Reader, Salesforce CDC Reader, and Salesforce Writer only

ServiceNow

OAuth

for use with ServiceNow Reader and ServiceNow Writer

Snowflake

  • Key Pair

  • OAuth

  • Username & password

for use with Snowflake Writer, Snowflake CDC Reader, Database Reader, and Incremental Batch Reader

Stripe

  • API Key

  • OAuth

for use with Stripe Reader

Zendesk

  • API token

  • OAuth

  • Username & password (Zendesk Creds)

for use with Zendesk Reader

Operational considerations for connection profiles

Monitoring connection profile usage

Connection profiles of certain endpoint types provide additional monitoring information on the usage of the connection profiles in applications.

To view the available metrics for a connection profile:

  1. Select Console.

  2. In the console window at the bottom of the page, enter mon <namespace>.<connection profile name>; (replacing <namespace> with the connection profile's namespace and <connection profile name> with the actual profile name) and click Execute.

The latest metrics for the connection profile will be displayed in the console output.

Alerts for connection profiles using OAuth

When using OAuth as the authentication method in a connection profile, you can set an alert when the Refresh Token is expiring.

Note

For certain endpoints, Refresh Tokens are configured to expire after a certain time period. This Refresh Token expiry is configured in the third-party application. The Refresh Token needs to be manually refreshed in the third-party application and the connection profile needs to be updated with the new Refresh Token.

Note

In this release, this alert type is supported only for connection profiles where Endpoint Type is Snowflake.

image1.png

Assume that at the time of enabling the alert, the time remaining for the Refresh Token expiry is n days. When the time remaining for Refresh Token Expiry is 20% of n, an alert is sent (based on the alert settings.

TokenAlert.png

On receiving this alert, you can update the connection profile by editing the connection profile, doing an OAuth Sign-in again and saving the connection profile. This will prevent any disruption to applications using this connection profile. The timer will be reset to zero once a new refresh token is received.

Best practices for connection profiles

For enterprises, securing credentials for external endpoints is a vital part of data governance. connection profiles help you achieve this by offering fine-grained permissions that create clear role separation and keep your credentials secure.

Configure read-only access to connection profiles

You can apply fine-grained permissions to the connection profiles to control who can use and edit them. For example, as a Striim admin, you can grant edit permissions to selected Striim users responsible for managing your organization’s access credentials, while providing read-only access to other users.

To grant read-only access of connection profiles to a set of users:

  1. Open the Striim Console from the main menu.

  2. Determine a non-admin role for the namespace that should be given read-only access to connection profiles. Grant the READ and SELECT actions for all CPs in this namespace to the selected role in the namespaces using the following command:

  3. GRANT READ, SELECT ON connectionprofile <namespace> TO ROLE <namespace>.<role_name>;

  4. Assign this role to the developers who will create Striim applications. They will now have required permissions to search and use all connection profiles in that namespace in their applications, but not to edit or delete the connection profiles.

Troubleshooting connection profiles

Handling OAuth refresh token updates

If a refresh token in a connection profile is revoked (for example, as part of a security-related token flush), a Striim admin must update the refresh token manually as follows:

  1. Stop all running applications that are currently using this connection profile.

  2. Update the refresh token in the connection profile.

  3. Test and save the connection profile.

  4. Restart the applications.