Feed and Parser related metrics are not available for the New Health and Wellness in RSA NetWitness Platform 11.5

Issue

The Feed and Parser metrics from the Network/Log Decoders are not available in the New Health & Wellness area. The metrics for Feeds and Parsers are disabled by default to keep the Health & Wellness system from hitting its default index field limit.

Resolution

This document is divided into several sections. Perform the following actions to enable the Feed and Parser metrics within the Network and Log Decoders using the New Health & Wellness system.

• Retrieving the Core Service UUID
• Retrieving the Include/Exclude Metrics Criteria for a Network/Log Decoder
• Updating the Include/Exclude Metrics Criteria for a Network/Log Decoder
• Adjusting Index Field Limits in the Health & Wellness UI

 

Retrieving the Core Service UUID

This article requires the retrieval of a Network or Log Decoder's Service ID within the NetWitness system. The following steps will show how to retrieve the Service ID for use in this document.

1. Log into the NetWitness UI and go to  (Admin Area) > Service and select a Network or Log Decoder.
2. Go to the Action icon for the service and click View > Explore.
3. Expand the sys node, and select the stats node.
4. In the left-hand panel look for UUID (uuid) and copy the id for later use. Ex UUID: 0089b3dd-715d-43ed-932b-e9e1777e3625

 

Retrieving the Include/Exclude Metrics Criteria for a Network/Log Decoder

1. SSH into the Admin Server (generally the server containing the UI).
2. Access the nw-shell program, by typing nw-shell at the prompt.

3. Connect to the metrics-server using connect --service metrics-server.

4. Type login to log into the metrics-server and use the NetWitness administrator account credentials.

5. Retrieve the current configuration for the Network Decoder or Log Decoder being adjusted and output it to a custom file on /root.

 

Updating the Include/Exclude Metrics Criteria for a Network/Log Decoder

Before proceeding with this section the Metrics-Server configuration for a Network/Log Decoder needs to be saved to a JSON file. See Retrieving the Include/Exclude Metrics Criteria for a Network/Log Decoder in this article before continuing. This section of the article will focus on adjustments to the Inclusion and Exclusion sections from the JSON file that was created earlier.

See the screenshot for an example of the output file that will be used.

1. Create a backup of the output file. Since the original file will be edited manually, it is important to have a clean backup should the original configuration need to be reapplied.
2. Edit the original output file in /root with vi.
   # vi /root/serviceConfig.json
3. Once editing the JSON file, to allow the New Health & Wellness to retrieve feed and parser statistics, remove the "/decoder/parsers/feeds/**" entry under the "exclusion" section. The ending coma for the line should also be removed.
   Original Exclusion Section
   "exclusion" : [
   "/decoder/config/rules/**",
   "/decoder/config/recovery/**",
   "/decoder/parsers/feeds/**", <--- This is the line in question
   "/sys/config/scheduler/**",
   "/sdk/stats/queries/**",
   "/decoder/devices/**"
   ],
   
   Adjusted Exclusion Section
   "exclusion" : [
   "/decoder/config/rules/**",
   "/decoder/config/recovery/**",
   "/sys/config/scheduler/**",
   "/sdk/stats/queries/**",
   "/decoder/devices/**"
   ],
   Note: When working with Core Services Inclusion/Exclusion rules, such as Network/Log Decoders, it is important to remember that the exclusion rules take precedence over the inclusion rules. In the JSON file output under the Inclusion section, it can be seen that /decoder/* is included, yet /decoder/parsers/feeds* is excluded which causes the parser and feeds to not provide statistics to the New Health and Wellness.
4. Save serviceConfig.json file in vi with :wq!
5. Access the nw-shell program, by typing nw-shell at the prompt.
   # nw-shell
6. Connect to the metrics-server using connect --service metrics-server.
   offline >> connect --service metrics-server
7. Type login to log into the metrics-server and use the NetWitness administrator account credentials.
   metrics-server:Folder:/rsa >> login
   user: admin
   password: ***********
   admin@metrics-server:Folder:/rsa >>
8. Set the configuration for the Network Decoder or Log Decoder to the new settings using the custom file in /root.
   admin@metrics-server:Folder:/rsa >> cd /rsa/metrics/elastic/set-config
   admin@metrics-server:Folder:/rsa/metrics/elastic/get-config >> invoke file /root/serviceConfig.json
9. The specific UUID referenced Network/Log Decoder is now updated to provide parser and feed statistics to the New Health & Wellness.

 

Adjusting Index Field Limits in the Health & Wellness UI

This section is designed to help adjust Indexing Field limits, via the nw-index-template, that can be reached after enabling additional statics on the Network/Log Decoders. The default index field limit in the New Health & Wellness is set to 20,000 fields. If there are errors in the /var/log/messages or elasticsearch.log, then the index field will need to be adjusted.

See the following log error example: To update the nw-index-template, follow these steps:

1. Log into the NetWitness UI.
2. Click the  (Admin Area) icon, then click Health and Wellness.
3. Click the New Health & Wellness tab and click on the Pivot to Dashboard button. This will open the new Deployment Health Overview
4. Click on the Dev Tools icon, which opens the browser-based Dev Tools Console.
5. Clear all content in the left panel and type in GET _template/nw-index-template and click the small right arrow (Run) to retrieve the existing nw-index-template.
6. Check the existing index limit from the right-hand panel. See the screenshot for and example where it is set to 20000. Make an external copy of the complete template for later use.

7. Click on Management, then click on Index-Patterns.
8. Click the nw* under the Index Pattern list.
9. At the top of the page will be Fields (#####) where the ##### represents the number of fields currently being indexed.
10. If the number from the Fields (####) is larger than the index limit retrieved in step 6, an index template adjustment will be required.
11. Go to the Dev Tools area.
12. Update the index limit in the nw-index-template that was copied in step 6. For example, changing the limit from 20000 to 28000.
13. Copy the template into the right-hand panel in the Dev Tools Console area.
14. In the left-hand panel type the following, PUT _template/nw-index-template and click the small right arrow (Run) to update the nw-index-template on the system.

15. Go to Management, then Index Patterns.
16. Click on the nw-decoder* pattern. Once it opens, click the  Refresh icon and then the Refresh button.
17. Go back to Index Patterns, click nw-logdecoder*. Once open, click the  Refresh icon and then the Refresh button.

At this point, the required field index limits have been increased for the Network and Log Decoders based on the new nw-index-template. If there are any questions or issues while using this article, please open an RSA NetWitness Support Case.

Product Details

RSA Product Set: NetWitness Platform
RSA Product/Service Type: Health & Wellness/Metrics Server
RSA Version/Condition: 11.5
Platform: CentOS
O/S Version: 7

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue