Striim Platform 5.0 documentation

Sentinel uses Striim AI to detect sensitive data in every input event and match it to a supported Sensitive Data Identifier. Sentinel's accuracy depends on the AI engines used, and these AI engines may occasionally misclassify information.

After detecting sensitive information in the input event, Sentinel can protect the information by encrypting or masking it before it flows further downstream. For example, in the figure below, assume that you have configured Sentinel with the following actions in accordance with your organization’s data governance policies (under Policy Actions on the Sentinel property panel):

As shown below, Sentinel protects sensitive information such as EMAIL_ADDRESS, CREDIT_CARD_NUMBER, and USA_SOCIAL_SECURITY_NUMBER by obfuscating them in the input event before publishing it to the output. Since you have not specified protective actions, i.e. encrypt or mask, for other Sensitive Data Identifiers, Sentinel will take no action when it detects sensitive information such as NAME, PHONE_NUMBER, ADDRESS, IND_AADHAAR_NUMBER, or GBR_NATIONAL_INSURANCE_NUMBER, and it will let such information pass through as is to the output.

The Actions on Data Identifiers depend solely on the type of sensitive information and not on its location in the input record. For example, in the input record in the figure below, a US Social Security Number has been mistakenly placed in the field named “Department”.  Since you have configured Sentinel to act on USA_SOCIAL_SECURITY_NUMBER, Sentinel is able to identify and partially mask this information, thereby preventing this sensitive information from flowing further downstream.

The Actions on Data Identifiers also support schema evolution. If new columns, fields, files, tables, collections or objects are added to your configured source, and thereby to your application, then Sentinel can automatically detect, identify and protect sensitive information in the newly added data, thus ensuring that your application remain governed through its lifecycle.

Sentinel can inform you about the sensitive data that it detected in the input record, and it can tag the input event by adding this information as AIData to the Striim event type (such as WAevent or JSONNodeEvent), as shown in the figure above.  AIData is conceptually similar to UserData (see: Adding user-defined data to WAEvent streams and Adding user-defined data to JSONNodeEvent streams), except that it is generated by Striim.  The output stream from Sentinel contains the event AIData, and you can query the output stream to learn about the sensitive data that Striim has detected in your record.  When you write the record to the target, you can also write the event AIData to the target so that you can know about the sensitive data that Sentinel detected in that record, else the AIData information for that record will be lost forever.

Your Striim Admin must classify the supported Sensitive Data Identifiers as "High Importance", "Medium Importance", or "Low Importance", in accordance with your organization's data governance policies. In your Sentinel configuration, you can specify the importance level of the Identifiers that Sentinel must tag in the event AIData. For example, in the figure above, assume that your Striim Admin has retained the default Sensitive Data Identifier importance classification, and you have configured Sentinel as shown. Accordingly, the event AIData contains the following information in JSON format:

Typical use cases for event tagging include the following:

You can encrypt the sensitive information using your Google KMS keys.  This action is similar to Encrypting data using Shield. 

Sentinel can completely mask the input sensitive information by replacing every character, including delimiters such as spaces, hyphens or commas, with “x” as shown in the examples below:

Sentinel can redact your sensitive information by masking all characters with “x” except for a fixed number of characters at the beginning and at the end of the information, as shown in the table below.  You must specify the number of characters that must be excluded from the masking.

If the number of characters in the input message is equal to or less than the sum of the total number of first and last characters that you want shown in the output message, then your input message will not be masked, as shown in the table below.

You can specify a regular expression (regex) pattern that Sentinel must search for and mask.  When you supply a regex pattern, Sentinel will replace each match with “x”, repeated for the length of the match.

For example, assume that your source contains US Social Security Numbers in the format 123-45-6789, and you want to mask these numbers while retaining the format.  You can provide a regex pattern such as (\\d{3})-(\\d{2}), and Sentinel’s output will be xxx-xx-6789 for sensitive information that it identifies as USA_SOCIAL_SECURITY_NUMBER  and that matches the supplied regex pattern.

Sentinel is available as a component in the flow designer palette.  You can place Sentinel in your application and configure it as shown below. You must set up the AI engines to work with Striim before you can run your application with Sentinel.

The following table provides examples of entities and fields for different sources so that you know how to fill the Actions on Fields section of the Sentinel property panel. For example, when you are reading from a relational database (RBDMS), you can think of the entity as a table in the source database, and the field as a column name in that table. Similarly, when you are reading from a NoSQL database such as MongoDB, you can think of the entity as a collection in the source database, and the field as the field name in the records in that collection.

The following is an example of a Sentinel component that detects and takes specific actions on identifiers.

CREATE OR REPLACE SENTINEL mySentinel DETECTION ON
  IDENTIFIER ACTIONS '{"AGE": {"action" : "MASK_COMPLETELY"}, 
  "EMAIL_ADDRESS": {"action" : "MASK_M_N", "m" : "2", "n" : "1"}}',
  entity Actions '{"AIDEMO.CUSTOMER": {"COL5" : {"action" : "MASK_COMPLETELY"}}}'
  TAGGING LEVELS 'HIGH,MEDIUM'
)
INPUT FROM sourceStream
OUTPUT TO PIIStream;

CREATE CQ putSentinelDataInUserData
INSERT INTO FinalStream
SELECT
DATA(e) data,
putUserData(e,'TaggedInfo', getAIData(e, 'Sentinel'))
FROM PIIStream e;