Resource usage policies
The core Striim application, your applications, and your users logged in and working on Striim — together, these consume the same set of CPU, memory and storage resources. How much Striim can do—how many applications it can run at a time, how many users can log in at once, and so on—is limited by the CPU cores, memory, and disk space available to it. If you try to do more than these available resources can handle effectively, it can lead to issues such as excessive CPU usage and out-of-memory errors, and these can ultimately degrade the performance of your Striim applications and environment. We have recommended the policy defaults discussed below to avoid accidentally running into such problems.
When a resource policy limit is reached, a ResourceLimitException is displayed in the web UI and logged in striim.server.log, and the action that exceeded the limit (such as deploying an application or logging in) fails.
Note
In this release, resource usage policies can only be viewed and managed using the console.
The default resource usage limits are in effect for all new Striim clusters. The resource usage limits are disabled for Striim clusters that you upgrade from previous versions. You can use the default usage policy limits, or adjust, and even disable, the resource usage limits as needed. You must log in with Striim admin credentials to be able to modify the resource usage policy limits.
Resource usage policy limits
Object/component | Default value and triggering action | Notes |
|---|---|---|
active_users_limit Number of active non-system users | 30 Limit checked: when you create a new user. | Scope: cluster A greater number of active users can mean that there are more applications, and may lead to overloading the system. (The system users, admin and sys, are not counted toward this limit.) |
api_call_rate_limit Rate limit for Rest API calls | 500 (per sec) Limit checked: when you make a REST API call. | Scope: server Serving the REST API calls takes resources on the server backend. Bursty REST API calls also are an indication of a user side uncontrolled application or at worst a DDOS type pattern. After altering / disabling this limit, you must restart the service. |
apps_per_cpu_limit Number of running applications based on CPU cores | 4 (applications per available core on the server) Limit checked: during deployment. | Scope: server Applications are the primary consumers of resources. To maintain a certain throughput level we need to limit the number of running applications as a function of the available vCPUs. |
apps_per_gb_limit Number of running applications based on memory | 2 (applications per 1 GB of memory available to the Java virtual machine running Striim) Limit checked: during deployment. | Scope: server The number of running applications is combination of both the CPU cores and memory limits. |
cluster_size_limit Number of servers and Forwarding Agents in the cluster | 7 Limit checked: when you add a new server or Forwarding Agent to the cluster. | Scope: cluster Benefits: The probability of a server failure increases as their number grows. |
num_queries_limit Number of adhoc and named (dashboard) queries | 50 Limit checked: when you run an ad-hoc query or a dashboard runs a named query. | Scope: server Too many unmanaged tasks/queries taking up system resources and destabilizing running applications will be limited. |
ui_sessions_limit Number of concurrent active web UI sessions | 10 Limit checked: when a user logs in through the UI. | Scope: server The Striim user interface is an active page which when loaded and open actively receives various types of data. Having too many of these pages open may lead to memory pressure. |
Application resource usage policies
Applications are the primary consumers of resources. Your Striim environment may be constrained by CPU resources or memory resources depending on the configuration of your underlying infrastructure. Thus, there are two separate policy limits that apply to the maximum number of applications that can concurrently run in your environment:
Number of running applications based on CPU cores
Number of running applications based on memory available to the Java virtual machine running Striim
The maximum number of concurrently running applications is determined by the combination of these two resource policy limits. That is, the limit will be a minimum of the number of applications that can run based on CPU and memory resources. For example, a server with 8 CPU cores and 16 GB memory can be considered to be constrained by memory resources. If you configure the application resource policies to allow a maximum of 1 application per GB of memory and 4 applications per CPU core, then Striim will allow a maximum of 16 applications to run at anytime because it is the lower of the 2 limits - 16 applications on the basis of memory and 32 applications on the basis of CPU cores.
Viewing resource policies
The following command shows the current value for a given resource limit policy:
describe resource_limit_policy <resourceLimitname>;
The following command lists all the names of the resource limit policies:
list resource_limit_policies;
Enabling or disabling resource policies as a group
You can enable or disable resource limits as a group using the alter cluster command:
alter cluster { enable | disable } resource_limit_policy;
After enabling or disabling resource limits, you must restart Striim before the change will take effect (see Starting and stopping Striim Cloud).
Disabling the resource_limit_policy turns off all resource limit checks.
Enabling the resource_limit_policy turns resource limit checks with the values of the limits reverting to those set by the user before disabling, or to the default values if no changes to the defaults have been made.
Modifying individual resource usage policies
You can enable or disable resource limits individually or change their values using the alter resource_limit_policy command. Policies apply to all servers in the cluster; you cannot set different policies for each server.
If you make a change to api_call_rate_limit, you must restart Striim before the change will take effect (see Starting and stopping Striim Cloud).
To disable an individual resource usage policy or several enumerated policies:
alter resource_limit_policy unset <"resourcelimitname">, ...;
For example, alter RESOURCE_LIMIT_POLICY unset "CLUSTER_SIZE_LIMIT", "APPS_PER_CPU_LIMIT";
To enable a resource usage policy or set a new value for the default limit:
alter resource_limit_policy set <"resourcelimitname">, ...;
For example, alter RESOURCE_LIMIT_POLICY set (CLUSTER_SIZE_LIMIT : 3, APPS_PER_CPU_LIMIT : 14);
The value must be a positive integer.