Striim 3.9.7 documentation

ALTER and RECOMPILE

Use these commands to modify applications that are loaded but not deployed or running (in other words, applications with status Stopped or Quiesced). Using ALTER instead of dropping and reloading retains persisted WActionStore and Kafka stream data. This could be useful, for example, when there is a DDL change to a source database table.

Note that some changes to the application can be made safely but others may be incompatible with persisted data, causing errors. For example:

should be compatible with persisted data

may be incompatible with persisted data

add components or flows

remove components or flows

add a field to a type

remove a field from a type

change a type from:

  • byte to integer, short, or long

  • integer to short or long

  • short to long

  • float to double

change a type from:

  • long to short, integer, or byte

  • short to integer or byte

  • integer to byte

  • double to float

Using ALTER when recovery is enabled (STOP vs. QUIESCE)

When you stop, alter, and restart an application with recovery enabled, it will resume operation with no missing or duplicate events ("exactly-once processing," also known as E1P), except as noted in Recovering applications and with the following possible excerptions:

should not interfere with exactly-once processing

may result in duplicate or missing events

  • adding components to an unbranched data flow (that is, to a series of components in which the output of each component is the input of only one other component)

  • removing components from an unbranched data flow

  • adding a branch to a data flow

  • removing a branch from a data flow

  • simple modifications to a CQ

  • modifying sources

  • changing a window's KEEP clause

  • changing a CQ's GROUP BY clause

  • changing the number of fields in a CQ's FROM clause

  • changing the size of a CQ's MATCH_PATTERN selection

  • changing the value of a CQ's LIMIT clause

  • changing a KafkaWriter's mode from sync to async

When you quiesce an application with recovery enabled, altering it in any way other than changing its recoverable sources will not interfere with exactly-once processing. However, there may be anomalous results (see the discussion of QUIESCE in Console commands for details).

Example

The workflow for ALTER and RECOMPILE is:

  1. If the application is running, STOP or QUIESCE it (see Console commands).

  2. Undeploy the application.

  3. Alter the application as described below.

  4. Recompile, deploy, and start the application.

To begin altering an application, use:

USE <application's namespace>;
ALTER APPLICATION <application name>;

At this point, enter CREATE, DROP, or CREATE OR REPLACE statements to modify the application. For example, to add the email subscription described in Sending alerts from applications to the PosApp sample application:

USE Samples;
ALTER APPLICATION PosApp;
CREATE SUBSCRIPTION PosAppEmailAlert
USING EmailAdapter (
  SMTPUSER:'sender@example.com',
  SMTPPASSWORD:'password', 
  smtpurl:'smtp.gmail.com',
  starttls_enable:'true',
  subject:"test subject",
  emailList:"recipient@example.com,recipient2.example.com",
  senderEmail:"alertsender@example.com" 
)
INPUT FROM AlertStream;
ALTER APPLICATION PosApp RECOMPILE;

Then complete the alteration with:

ALTER APPLICATION <application name> RECOMPILE;

At this point you may deploy and start the modified application. If recovery was enabled for the application when it was loaded, when it is restarted, it will pick up source data (subject to the usual limitations detailed in Recovering applications) back to the time it went offline.

Keep in mind that a change made to one component may require changes to multiple downstream components and their types. For example, consider .../Striim/Samples/RetailApp/RetailApp.tql. Its OrderType contains city and state fields:

CREATE TYPE OrderType(
  storeId      String,
  orderId      String,
  sku          String,
  orderAmount  Double,
  dateTime     DateTime,
  hourValue    Integer,
  state        String,
  city         String,
  zip          String
);
CREATE STREAM RetailOrders Of OrderType;

The city and state fields can be retrieved downstream when creating TrackStoreActivity via a SELECT statement that gets the fields ZipCodeLookup. This means that the city and state fields can be excluded from several earlier statements. We will use CREATE OR REPLACE commands for the following portions of the RetailApp TQL:

  • OrderType

  • ParseOrderData

  • StoreOrdersTrackingType

  • GetStoreActivity

  • GetStoreStatus

  • TrackStoreActivity

The following TQL can be used to update the sample:

USE Samples;
ALTER APPLICATION RetailApp;
CREATE OR REPLACE OrderType (
  storeId      String,
  orderId      String,
  sku          String,
  orderAmount  Double,
  dateTime     DateTime,
  hourValue    Integer,
  zip          String
);
CREATE OR REPLACE CQ ParseOrderData
INSERT INTO RetailOrders
SELECT  data[0],
        data[6],
        data[7],
        TO_DOUBLE(SRIGHT(data[8],1)),
        TO_DATE(data[9],'yyyyMMddHHmmss'),
        DHOURS(TO_DATE(data[9],'yyyyMMddHHmmss')),
        data[4]
FROM Orders;
CREATE OR REPLACE TYPE StoreOrdersTrackingType (
  storeId String KEY,
  zip   String,
  startTime DateTime,
  ordersCount Integer,
  salesAmount Double,
  hourlyAvg Integer,
  upperLimit Double,
  lowerLimit Double,
  category String,
  status String
);
CREATE OR REPLACE CQ GetStoreActivity
INSERT INTO StoreOrdersTracking
SELECT rd.storeId, rd.zip, first(rd.dateTime),
       COUNT(rd.storeId), SUM(rd.orderAmount), l.hourlyAvg/6,
       l.hourlyAvg/6 + l.hourlyAvg/8,
       l.hourlyAvg/6 - l.hourlyAvg/10,
       '<NOTSET>', '<NOTSET>'
FROM RetailData_5MIN rd, HourlyStoreSales_Cache l
WHERE rd.storeId = l.storeId AND rd.hourValue = l.hourValue
GROUP BY rd.storeId;
CREATE OR REPLACE CQ GetStoreStatus
INSERT INTO StoreOrdersTracking_Status
SELECT storeId, zip, startTime,
       ordersCount, salesAmount, hourlyAvg, upperLimit, lowerLimit,
       CASE
         WHEN salesAmount > (upperLimit + 2000) THEN 'HOT'
         WHEN salesAmount > upperLimit THEN 'MEDIUM'
         WHEN salesAmount < lowerLimit THEN 'COLD'
         ELSE 'COOL' END,
       CASE
         WHEN salesAmount > upperLimit THEN 'TOOHIGH'
         WHEN salesAmount < lowerLimit THEN 'TOOLOW'
         ELSE 'OK' END
FROM StoreOrdersTracking;
CREATE OR REPLACE CQ TrackStoreActivity
INSERT INTO StoreActivity
SELECT s.storeId,
  s.startTime,
  n.storeName,
  s.category,
  s.status,
  s.ordersCount,
  s.salesAmount,
  s.hourlyAvg,
  s.upperLimit,
  s.lowerLimit,
  z.zip,
  z.city,
  z.state,
  z.latVal,
  z.longVal
FROM StoreOrdersTracking_Status s, StoreNameLookup n, ZipCodeLookup z
WHERE s.storeId = n.storeId AND s.zip = z.zip
LINK SOURCE EVENT;
ALTER APPLICATION RetailApp RECOMPILE;