Event Stream Analysis (ESA) rule is disabled after being deployed in NetWitness 10.6 and higher

Issue

In 11.X version, An ESA rule is disabled after being deployed to the ESA service and reports the error below.

ESA was unable to deploy one or more rules, and these rules were disabled. Common issues include: missing metadata, invalid rule syntax, and unavailable external connections at the time of deployment.

The ESA log level WARN contains the following message:

 Implicit conversion from datatype 'String' to 'String[]' is not allowed

In 12.3 version, The deployed rule with an array error, shows the error in the UI under the ESA Deployment stats. Navigate to Policies>Content>hover over the policy status and it will show below errors.

ESA /var/log/netwitness/correlation-server/correlation-server.log shows below errors:

  Caused by: com.espertech.esper.compiler.client.EPCompileException: Failed to validate filter expression '"google"=any(alias_host) or alias_h...(51 chars)': Collection or array comparison and null-type values are not allowed for the IN, AN
  
Y, SOME or ALL keywords [@RSAAlert(oneInSeconds=0)         SELECT * FROM Event(             /* Statement: finding array */             (( 'google' = ANY( alias_host ) ) OR alias_host IN ( 'login' ))                      )]

Cause

Within the ESA service, some meta keys were changed from a string type to a multi-valued type. This affected the following rules:

Column 1: Rule #
Column 2: Rule Name
Column 3: Array Type Meta Keys in 11.3

Column 1: 1
Column 2: RIG Exploit Kit
Column 3: threat_category

Column 1: 2
Column 2: AWS Critical VM Modified
Column 3: alert

Column 1: 3
Column 2: Multiple Successful Logins from Multiple Diff Src to Same Dest
Column 3: host.src and host.dst

Column 1: 4
Column 2: Multiple Successful Logins from Multiple Diff Src to Diff Dest
Column 3: host.src and host.dst

Column 1: 5
Column 2: Multiple Failed Logins from Multiple Diff Sources to Same Dest
Column 3: host.src and host.dst

Column 1: 6
Column 2: Multiple Failed Logins from Multiple Users to Same Destination
Column 3: host.src and host.dst

Column 1: 7
Column 2: User Login Baseline
Column 3: host.src and host.dst

Resolution

To change the string type meta keys to string array type meta keys in NetWitness Platform 11.3, see “Configure Meta Keys as Arrays in ESA Correlation Rule Values” in the ESA Configuration Guide for RSA NetWitness® Platform 11.3

NetWitness 11.3:

To deploy custom ESA rules using the above listed meta keys, the rules must be updated to use the array syntax and then redeployed. For example:

Column 1: String Syntax
Column 2: Array Syntax

Column 1: threat_category = 'rig'
Column 2: 'rig' = ANY(threat_category)

If you had any of the above listed rules deployed before 11.3, note any rule parameters that you have changed in order to adjust the rules for your environment. Download the updated rules from Live. Reapply any changes to the default rule parameters and deploy the rules. (For instructions, see “Download RSA Live ESA Rules” in the Alerting with ESA Correlation Rules User Guide for RSA NetWitness® Platform 11.3

NetWitness 11.2 and Prior:

To deploy Live ESA rules using these keys, the meta keys must be added to the ESA service using the multi-valued type. In addition, any custom ESA rules using these meta keys must be updated to use array syntax. The steps below explain how to add the meta keys to the ESA service with the multi-valued type.

In the NetWitness UI, go to Administration > Services > ESA > Explore > Workflow > Source > nextgenAggregationSource > ArrayFieldNames.

In the ArrayFieldNames property, enter the meta keys separated by commas. Be sure to use underscores for multi-word meta keys.
Restart the ESA service using below command.

 service rsa-nw-correlation-server restart

Product Details

NetWitness Product Set: NetWitness Platform
NetWitness Product/Service Type: Event Stream Analysis (ESA)
NetWitness Version/Condition: 11.3 and Higher version
Platform: CentOS 7 / Alma

Approval Reviewer Queue

Technical approval queue