Skip to content
  • There are no suggestions because the search field is empty.

Rule Syntax

Rule SyntaxRule Syntax

This section describes the different rule syntax supported in the Reporting Engine.

NWDB Rule SyntaxNWDB Rule Syntax

The NWDB rule is one of the rule syntax supported in the Reporting Engine. To enhance the execution time of your reporting entities, see "Reporting Guidelines" section in Reporting Overview.

A Rule is a function that manipulates the result set of a rule in order to make the output in a report more meaningful or add additional functionality to a rule other than querying data and displaying it. Any combination of these rule actions can be used to create unique and interesting representations of the information collected by NetWitness.

The Reporting Engine supports the following categories of NWDB data source rule syntax:

  • select clause

    • Non-Aggregate Rule
    • Aggregate Rule
  • alias
  • where clause
  • where clause Operators
  • then clause
  • Limit field
  • Rule Actions
  • Rule Operators

Select Clause

The select clause is a comma separated list of values. For example: select sessionid,time,service.

​There are two types of select clause for NWDB Rule:

  • Non-aggregate rule
  • Aggregate rule

Non-Aggregate Rule

When you want to define a rule without any grouping, choose 'None' in the Summarize field. In a non-aggregate rule, you can select any number of metas in the Select clause. For example, select service, sessionid, time.

netwitness_110_non_aggregate_fn.png

Aggregate Rule

When you want to query for a specific meta and its associated aggregate value then you must use the Aggregate rule. To get an aggregate, you must choose either of the three metas (Event Count, Packet Count, Session Size) or choose 'Custom' in the Summarize field to include an aggregate function in the Select clause. For example, select ip.src, sum (ip.dst). When Custom aggregate rule is enabled, the following fields are populated in the user interface:

  • Group By
  • Order By
  • Session Threshold

The following figure shows the Build Rule view for Aggregate Rule.

netwitness_110_agr_rule.png

There are two types of aggregate values that can be queried:

  • Collection aggregation
  • Meta aggregation

Collection Aggregation

With collection aggregation, you can get aggregates related to Event, Session or Packets. The following values can be queried in a collection aggregation:

  • Event Count: The total count of events.
  • Packet Count: The total count of packets.
  • Session Size: The total session size.

These options are listed in 'Summarize' field and any one of them can be selected in a rule.
For example, choose any of the Collection aggregates (Event Count or Packet Count or Session Size) in the 'Summarize' field and select ip.src.

netwitness_110_agg_fn_event_count.png

Meta aggregation

With meta aggregation, you can get aggregates of meta values. The following are the supported meta aggregate functions:

  • sum(meta)
  • count(meta)
  • countdistinct(meta)
  • min(meta)
  • max(meta)
  • avg(meta)
  • first(meta)
  • last(meta)
  • len(meta)
  • distinct(meta)

Supported Meta Aggregate Functions

The NWDB service supports the following meta aggregate functions and syntax in this release.

You must select 'Custom' in 'Summarize' field and provide the meta and the meta aggregate functions in the select clause.
netwitness_110_agg_fn_custom.png

Note: Meta aggregate functions cannot be used in a WHERE clause and the rule actions like min_threshold/max_threshold can be used to filter aggregate functions. It is advised to use a more refined WHERE clause to get a better rule performance while using 'group by'.

Aggregate Query for Multiple Meta

To execute aggregate query for multiple Meta, follow these steps:

  1. Go to Reports.

    The Manage tab is highlighted and the Rules view is displayed.

  2. In the Rules toolbar, click netwitness_110_add_button.png > NetWitness Platform DB.

    For example, enter the following meta in the fields highlighted below:

    SELECT: ip.src, service, count(alias.host)
    ALIAS: Source IP Address, Service Type, count(alias.host)
    WHERE: ip.src = 59.96.136.142

    Note: In the alias field you can enter a name for columns used in the select clause. If you do not specify the alias for one of the field in the select clause, then the default description will be used. For example, if the select clause has Field1, Field2, Field3, Field4, and alias has only Field1, Field3, Field4, then for Field2 a default description is used.

  3. Click the Test Rule button at the bottom of the screen.

    The Test Rule page is displayed.

    netwitness_110_multiple_qry_meta.png

Summarize

Summarize determines the type of summarization or aggregation for the rule.

Order By

Order By determines how to sort the result set.

Session Threshold

The session threshold is the optimization setting to stop scanning the matching sessions for each possible
unique value for the selected meta. The threshold is an integer between 0 (default) and 2147483647. The threshold 0 scans for all matching sessions.

Note: If you provide a non-zero value (a value higher than zero), the aggregate results are inaccurate. This can be used only when you are interested in unique values and not aggregate values.

Supported where Clause

In the where clause, make sure the syntax is correct based on the meta type.
For example,
For all text meta type use quotes for example, username = ‘user1’.
For all IP Addresses, Ethernet Addresses, and Numeric meta types do not use quotes for example, service = 80 && ip.src = 192.168.1.1.
For date and time meta types, if the date and time format is 'YYYY-MM-DD HH:MM:SS‘, use quotes.
If the date and time format is 1448034064 (number of seconds since EPOCH (Jan 1, 1970)), do no use quotes.

Note: If list is used in the rule, make sure that the list values are quoted or unquoted based on the type of the meta used. Checking the Quotes will be inserted for all the values checkbox in list definition page (for more information see, "Create Lists or List Groups" section in Configure a Rule) would quote all the list values. ​

Supported where Clause Operators

Supported then Clause

Limit field

This indicates the limit to be put on the query while fetching data from the database. If a result set is sorted by event
count, packet count, or session size, the limit represents the top (or bottom) N values to be returned. If the result set is not
sorted, the first N values are returned.

Rule Actions

The NWDB data source rule syntax supports the following rule actions:

  • dedup
  • filter_on
  • filter_out
  • lookup_and_add
  • max_threshold
  • min_threshold
  • regex
  • sum_count
  • sum_values
  • show_whats_new

dedup (string field)

dedup removes the duplicate entries in an unsorted result set and displays only pertinent data. The dedup rule action removes duplicate entries of a specific field in the report, so that only the first occurrence of that value is listed in the report.

Note: The dedup rule action cannot be used with an aggregate rule.

For example, the meta data generated by an individual session is often repetitive, especially when you have sessions with a lot of DNS lookups or web sessions that access the same host multiple times for various resources (such as, javascript, css). To remove the duplicate entries of the host, you can use the dedup rule action.

Example:

The following example is a lengthy result set that can be trimmed by removing the duplicate values in the same session.

netwitness_110_dedup_before.png

The following figure shows the use of dedup rule action to remove the duplicate entries from the result set.

netwitness_110_dedup_build.png

The duplicate value for each entry in the rule result set is reduced to one value.

netwitness_110_dedup_after_849x508.png

filter_on (string filter, string field, bool matchExact)

filter_on removes values that do not contain the filter criteria from the result set. If the result set contains multiple fields, you must select a specific field to which the filter is applied. To add additional results to a single result set, include function such as lookup_and_add.

The matchExact parameter determines if the match is an exact match or contains a match.

  • If matchExact is set to false, any value that contains the filter text is considered a match.
  • If matchExact is set to true, only values that match the provided filter text is included in the result set.

Note: Unless the matchExact parameter is specified, the default behavior of the rule action is to match exactly the text specified in the filter parameter. To specify that results containing the filter text must be kept in the result set, users must set the matchExact parameter to false.

Example:

The following figure displays the list of countries and their event count.

netwitness_110_before_filteron_result.png

The following figure shows a filter_on rule action to filter out countries except Spain, China, United States and United Kingdom from the result set.

netwitness_110_with_filteron_build_rule.png

The following figure shows the output with the filter_on rule action.

netwitness_110_with_filteron_result.png

Another way of filtering out the entries from the result set is to create a list of variables which you want to filter out. For example, you can create a list with United Kingdom, France and Germany as values in the list. You can use this list in the rule action to get the same result set. For example, if you create a list called COUNTRY_LIST, you can use the list as follows:

filter_on ('$COUNTRY_LIST', 'country.src', 'false');

filter_out (string filter, string field)

filter_out (string filter, string field, bool matchExact)

filter_out removes the values that contain the filter criteria from the result set. If the result set contains multiple fields, you must select a specific field to which the filter is applied (for example, you can use a lookup_and_add to add results to a single result set).

The matchExact parameter determines if the match is an exact match or contains a match.

  • If matchExact is set to false, any value that contains the filter text is considered a match.
  • If matchExact is set to true, only values that match the provided filter text is excluded from the result set.

Note: Unless the matchExact parameter is specified, the default behavior of the rule action is to match exactly the text specified in the filter parameter. To specify that results containing the filter text must be removed from the result set, users must set the matchExact parameter to false.

Example:

The following figure displays the list of countries and their event count.

netwitness_110_before_filter