Monitoring the status and health of Striim servers

Certain Striim solutions such as Striim Cloud Mission Critical provide tools to deploy and run applications on multiple servers for higher performance or availability. Monitoring the health of the cluster and individual Striim servers allows you to plan application deployments accordingly. You can quickly see the overall health of the cluster through the Status and Health Status columns.

Health status

The following are the possible status for the Health Status column:

Healthy: The Striim cluster is operating normally. All user-specified Striim servers are running.
Degraded: Some nodes in the Striim cluster are not working properly, but the cluster performance is not affected. Striim staff is investigating. Numerically, this status is shown when at least one node is down but the number of running nodes is above half those specified by the user.
Critical: There are insufficient healthy nodes for cluster performance. Expect increased processing latency. Striim staff is actively addressing the issue. Numerically, this status is shown when the number of nodes down is less than half specified by the user.
Offline: The cluster is unavailable. Striim staff is attempting to restore functionality.

Health status indictors are shown only when the service is in the Running state. Striim Cloud Mission Critical users are not expected to take any corrective action when you see a Degraded, Critical, or Offline health status. Striim is notified about the degradation and will investigate the issue.

Single server Striim solutions display only the Healthy or Offline statuses.

Service status

By comparison, the Status column covers actions under user control. There are the following possible statuses:

Creating: The service is being created.
Starting: The service has been created and is starting.
Running: The service is running.
Stopped or Stopping: The service has been stopped by the user, or is in the process of stopping.
Upgrading: The service is being upgraded.
Resizing: The service is being resized to have more Striim servers or to increase the capacity of the VM used.

Using observability tools with Striim Cloud

Striim Cloud exposes application metrics that you can process with your existing third-party observability tools, such as Dynatrace. Striim uses an Open Telemetry collector to export metrics from Striim Cloud to a telemetry backend. You can leverage your observability tools to help monitor and analyze the performance of your Striim applications in the Striim Cloud environment. By analyzing the data you can gain insights into application performance and reliability, and make any necessary changes to your application to improve performance.

The following observability tools are currently supported:

Dynatrace
Google Managed Service for Prometheus (GCP only)
New Relic
Prometheus Remote Write

Endpoint URL formats

Use the following endpoint URL formats when configuring each observability tool.

Observability tool	Endpoint URL format
Dynatrace	`https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/metrics`
Google Managed Service for Prometheus	Not applicable. GMP uses GCP IAM-based authentication instead of an endpoint URL and API token.
New Relic	`https://metric-api.newrelic.com` (US region) `https://metric-api.eu.newrelic.com` (EU region)
Prometheus Remote Write	`https://<prometheus-endpoint-domain>/api/prom/push` Use the Grafana Cloud or Prometheus-compatible endpoint assigned to your environment.

Configuring an observability tool

The Striim admin can configure an observability tool at the account level, and add new observability tools as they are supported. Users can choose to use an observability tool with an existing service or while creating a new service. The metric data will then be streamed into the associated observability tool. For example, your Striim admin can configure Striim by associating your Striim Cloud service with your Dynatrace account. Your users can then choose to use the tool with both existing or new services.

To configure an observability tool at the account level (Striim admin) and associate it with a service:

The Striim admin navigates to the Services page.
From the Services page, the Striim admin selects More and Deploy observability tool from the More menu of a service.
If there are no tools configured, click Add to configure a new tool. Or create a new tool from the Observability tools page.
Specify the following configuration for the tool:
- Name: a description name for your tool. You will be specifying this name when associating the tool with Striim service(s).
- Select observability tool name: choose your tool from the drop down menu.
- Endpoint URL: The URL for your observability tool endpoint. For example, for Dynatrace the URL of the endpoint is in the form https://example.live.dyntrace.com/api/config/v1/plugins/custom.remote.python.sap/endpoints/<endpoint-id>. For more information, see the documentation for your observability tool.
- API token: The API token for sending authenticated requests to your observability tool endpoint.
You can now see the tool appear in the list of observability tools for the account. It can be associated with one or more Striim services.

Associating an observability tool with a Striim service

Admins can choose to associate an observability tool with an existing service, or after creating a new service. If there are no available observability tools to deploy, contact your Striim admin.

To associate an observability tool with a Striim service:

The Striim admin navigates to the Services page.
From the More menu of a service, select Deploy observability tool.
Select the observability tool from the drop down menu, and choose Save.

Deleting an observability tool

To delete an observability tool, you must first disassociate the tool from all services.

To do so, select Disassociate observability tool in the configuration for each associated service.

Metrics available through open telemetry

Striim streams the following metrics to the associated observability tool.

Striim node health
Application throughput and latency
Selected adapter-specific behavior

Use the downstream metric name to locate each metric in your observability tool.

Some metrics are reported only by specific adapters, such as MongoDB Reader and Azure Event Hub Writer. Metrics that are broken out by attribute, such as collection name or topic partition, appear in your observability tool as separate time series.Azure Event Hub Writer

Component or adapter	Striim metric name	Metric name in the observability tool	Type	Description	Example
Striim node	`MEMORY_USED_PERCENT`	`memory_used_percent`	Gauge, percentage	Percentage of the node's total memory in use. Reported per server.	54.8
Striim node	`CPU_PER_CORE_PCT`	`cpu_per_core_pct`	Gauge, percentage	Average CPU utilization per core. Reported per server.	10.0
Striim node	`DISK_FREE`	`disk_free`	Gauge, percentage	Free disk space on the node, as a percentage of total disk space.	50.0
Striim node	`UPTIME`	`uptime`	Gauge, timespan	Time elapsed since the node last restarted.	1h:30m:05s
Striim node	`VERSION`	`version`	String	The Striim build running on the node.	4.2.0.2
All components	`INPUT`	`input`	Counter	Total events read into the component since the application started.	1000
All components	`OUTPUT`	`output`	Counter	Total events written out of the component since the application started.	1000
Target adapters	`ACCEPTED`	`accepted`	Counter	Events accepted by the target.	1000
Target adapters	`TARGET_OUTPUT`	`target_output`	Counter	Events successfully sent downstream to the target.	1000
Target adapters	`TARGET_ACKED`	`target_acked`	Counter	Events acknowledged by the target adapter.	1000
File adapters, Oracle Reader, MongoDB Reader	`AVG_IO_TIME`	`avg_io_time`	Gauge, milliseconds	Average time spent on I/O during the most recent monitoring interval. For MongoDB Reader, the time spent waiting for an available socket while sending data.	125
MongoDB Reader	`AVERAGE_EVENTS_PER_COLLECTION`	`average_events_per_collection`	Gauge, events per second	Average events fetched per second, reported per collection using the `collection-name` attribute. Initial load and incremental mode are reported together.	250.5
Application	`LAG_END2END`	`lag_end2end`	Gauge, seconds	End-to-end lag from the source read to the target write, reported per source pipeline using the `source` attribute.	0.5
Application	`STATUS_CHANGE`	`status_change`	Gauge, 0 or 1	Set to 1 while the application is running, and to 0 when it is in the CRASH, HALT, NOT_ENOUGH_SERVERS, UNKNOWN, DEPLOY_FAILED, or TERMINATED state. Applications in the system namespace are not reported.	1
Azure Event Hub Writer	`ACTIVE_EVENTHUB_CONNECTIONS`	`eventhub_active_connections`	Gauge, count	Total connections currently open to the event hub.	3
Azure Event Hub Writer	`TOPIC_PARTITION_INFO`	`no_of_records_in_previous_batch`	Gauge, count	Records sent in the previous batch, reported per topic partition.	120
Azure Event Hub Writer	`TOPIC_PARTITION_INFO`	`request_size_of_previous_batch`	Gauge, MB	Payload size of the previous batch, reported per topic partition.	2
Azure Event Hub Writer	`TOTAL_NO_OF_RECORDS_SENT`	`total_no_of_records_sent`	Counter	Total records successfully sent to the event hub.	10,000
Azure Event Hub Writer	`TOTAL_NO_OF_RECORDS_RETRIED`	`total_no_of_records_retried`	Counter	Cumulative count of record sends that had to be retried.	0
Azure Event Hub Writer	`DISCARDED_EVENT_COUNT`	`discarded_event_count`	Counter	Events discarded rather than delivered.	0
Azure Event Hub Writer	`AUTHORIZATION_EXCEPTION`	`authorization_exception`	Counter	Count of authorization and authentication failures.	0
Azure Event Hub Writer	`NETWORK_EXCEPTION`	`network_exception`	Counter	Count of network-level connection failures.	0

Note

For more information on monitoring in Striim, see the Monitoring Guide.