Skip to main content

Connecting Striim Cloud on AWS to self-managed data services

This topic describes how to enable private (non-public) connectivity from Striim Cloud hosted on AWS to customer databases hosted in Google Cloud Platform (GCP), Microsoft Azure, Oracle Cloud Infrastructure (OCI), and on-premises data centers. The connection leverages AWS VPC Endpoint Service via the customer's AWS account to ensure secure, private communication between Striim Cloud and your data sources without exposing traffic to the public internet.

If your database is on-premises or in another cloud provider (GCP, Azure, OCI), you must first establish VPN or Direct Connect connectivity to your AWS VPC. If your database is already within AWS, this step is not required.

The following diagram shows the connection architecture for self-managed databases hosted on AWS.

aws-pl-self-managed-1.png

The following diagram shows the connection architecture when connecting through VPN to databases in on-premises data centers or other cloud providers.

aws-pl-self-managed-onprem-cc-2.png

Prerequisites

  • A VPC network with at least four subnets in four different availability zones.

  • AWS IAM Administrator role for the user.

  • AWS account number that is hosting the Striim service. Contact your Striim team to obtain this account number.

  • (Optional, for cross-cloud or on-premises connections only) A VPN, Express Route, Direct Connect, or similar connection between your customer AWS VPC and your on-premises or cross-cloud network.

Prepare customer VPN with on-premises or cross-cloud network

If you need to connect to databases in on-premises data centers or other cloud providers, you must first establish connectivity between your AWS VPC and the target network.

For AWS to on-premises or cross-cloud connections, see Get started with AWS Site-to-Site VPN in the AWS documentation.

Launch an EC2 forwarder instance

You must launch an EC2 instance in your AWS VPC that will forward traffic from the load balancer to your database.

  1. In the EC2 console, launch a new EC2 instance.

  2. Choose Ubuntu as the operating system type.

  3. In the Network settings, choose the correct VPC, subnet, and default security group. Optionally enable assign public IP during creation.

  4. Select an m6a.large instance type or larger. This instance size is sufficient for this solution.

  5. Once the instance is ready, add a VPC Security Group Inbound Rule to accept SSH connections from your IP address.

Configure IP forwarding rules for Ubuntu

You must configure IP forwarding on the forwarder instance to route traffic from the load balancer to your database. This configuration assumes you can successfully access your database from the forwarder VM over VPN, Direct Connect, or locally.

  1. Connect to the forwarder VM using SSH.

  2. Execute the following commands as root user to create IP forwarding rules:

    # local port where Load Balancer sends traffic to:
SOURCE_PORT1=<portT1>
# destination server inside internal network:
DESTINATION_IP1=<databaseT1_IP>
DESTINATION_PORT1=<portT1>

# enable IP forwarding on host:
echo 1 > /proc/sys/net/ipv4/ip_forward

# clear existing iptables rules and chains:
iptables -F
iptables -t nat -F
iptables -X

# change the packet recipient from local to destination socket (host & port):
iptables -t nat -A PREROUTING -p tcp --dport ${SOURCE_PORT1} -j DNAT --to-destination ${DESTINATION_IP1}:${DESTINATION_PORT1}

# change the source IP address from the LB NAT IP address to the IP of this LB-backend host:
iptables -t nat -A POSTROUTING -p tcp -d ${DESTINATION_IP1} --dport ${DESTINATION_PORT1} -j SNAT --to-source $(hostname -i)

# to save iptables rules:
iptables-save

# Additionally, to make this rule persistence during reboot download iptables-persistent package (press yes in the popup window to acknowledge.):
apt install iptables-persistent

# Additionally, to review iptables rules:
iptables -t nat -nvL

  3. Replace the placeholder values with your actual destination port and IP address. The source port must be the same port as the destination port. The destination IP and port are your database IP address and port number.

Create additional forwarding rules

If you need to forward traffic to multiple databases, you can create additional forwarding rules without clearing the existing iptables rules.

# local port where Load Balancer sends traffic to:
SOURCE_PORT2=<portT2>
# destination server inside internal network:
DESTINATION_IP2=<databaseT2_IP>
DESTINATION_PORT2=<portT2>

# Do not clear existing iptables rules and chains

# change the packet recipient from local to destination socket (host & port):
iptables -t nat -A PREROUTING -p tcp --dport ${SOURCE_PORT2} -j DNAT --to-destination ${DESTINATION_IP2}:${DESTINATION_PORT2}

# change the source IP address from the LB NAT IP address to the IP of this LB-backend host:
iptables -t nat -A POSTROUTING -p tcp -d ${DESTINATION_IP2} --dport ${DESTINATION_PORT2} -j SNAT --to-source $(hostname -i)

# to save iptables rules:
iptables-save

# Additionally, to review iptables rules:
iptables -t nat -nvL

Configure IP forwarding rules for Windows

If you are using a Windows instance as the forwarder, you can create IP forwarding rules using netsh. This configuration assumes your Windows firewall is disabled.

  1. Open a CMD terminal on the Windows instance.

  2. Create IP forwarding rules using netsh:

    netsh interface portproxy add v4tov4 listenport=<port_number> listenaddress=<ip_address> connectport=<port_number> connectaddress=<target_IP>

  3. Verify the port forwarding rule:

    netsh interface portproxy show all

Create a load balancer target group

You must create a target group in the same VPC as the forwarder instance to route traffic from the load balancer to the forwarder.

  1. In the EC2 console, navigate to Target Groups.

  2. Click Create target group.

  3. Select Instances as the target type.

    aws-pl-screen-1.png

  4. Enter a name for the target group.

  5. For the port, enter the same port number as your database port.

  6. Select the VPC that contains your forwarder instance.

    aws-pl-screen-2.png

  7. Register the forwarder instance that you created as a target.

    aws-pl-screen-3.png

  8. Click Create target group.

Create an internal network load balancer

You must create an internal Network Load Balancer to send traffic from the VPC endpoint to the target group.

  1. In the EC2 console, navigate to Load Balancers.

  2. Click Create load balancer.

  3. Choose Network Load Balancer.

  4. Enter a name for the load balancer.

  5. For Scheme, select Internal.

  6. Select the VPC that contains your forwarder instance.

    aws-pl-screen-4.png

  7. Select your VPC security group.

  8. Add a listener with protocol TCP and port number matching your database port.

  9. For the default action, select the target group you created.

  10. Click Create.

    aws-pl-screen-5.png

Create a VPC endpoint service

You must create a VPC Endpoint Service that Striim Cloud will use to establish a private connection to your network.

  1. In the VPC console, navigate to Endpoint Services.

  2. Click Create endpoint service.

  3. Enter a name for the endpoint service.

  4. For Load balancer type, select Network Load Balancer.

  5. Select the load balancer that you created for this setup.

    aws-pl-screen-6.png

  6. For Acceptance required, select Acceptance required. This ensures you must manually approve connection requests.

  7. Click Create.

    aws-pl-screen-7.png

  8. After the endpoint service is created, navigate to the Allow principals tab.

    aws-pl-screen-8.png

  9. Add the Striim AWS account ID as an allowed principal. Use the account ID you obtained in the prerequisites.

  10. Copy the VPC Endpoint Service Name. You will need this value in the next step.

    aws-pl-screen-9.png

Create VPC endpoint from Striim service portal

You must create a VPC endpoint in the Striim service portal to initiate the connection request to your VPC endpoint service.

  1. Log in to your Striim account.

  2. Select the service for which you want to create the endpoint.

  3. Under the Secure connection tab, click Create PrivateLink.

  4. In the popup dialog, enter a name for this endpoint. Use a concise and meaningful name.

  5. In the Service name field, enter the VPC Endpoint Service name that you copied from the previous step.

  6. Leave the Target type field blank.

  7. Click Create.

    aws-pl-screen-10.png

  8. The endpoint displays a status of Pending initially while it waits for your acceptance in the AWS console.

    aws-pl-screen-11.png

Accept VPC endpoint connection request

You must manually accept the VPC endpoint connection request in the AWS console before the connection becomes active.

  1. In the VPC console, navigate to Endpoint Services.

  2. Select the endpoint service you recently created.

  3. Click the Endpoint connections tab.

  4. Select the endpoint connection that is in Pending state.

  5. From the Actions dropdown menu, select Accept endpoint connection request.

    aws-pl-screen-12.png

  6. Within a few minutes, the connection status changes to Available, indicating your endpoint connection is ready to use.

    aws-pl-screen-13.png

Note

You can reject endpoint connection requests at any time by selecting Reject endpoint connection request from the Actions dropdown menu. This disconnects the connection.

Use the endpoint connection for your data source

After the endpoint connection is available, you can use the endpoint FQDN to configure your Striim data source connection.

  1. In your Striim account, navigate to the service and select the Secure connection tab.

  2. Copy the FQDN of the newly created endpoint connection.

    aws-pl-screen-14.png

  3. Use this FQDN as the connection string for your data source in your Striim application.

Endpoint for MongoDB Atlas

For information about connecting to MongoDB Atlas, see the following topic.