Skip to content
  • There are no suggestions because the search field is empty.

NetWitness AZURE monitor collection stopping frequently due to malformed logs from Event source

Issue

AZURE Monitor logs collection stops frequently with below errors. The collection starts automatically after some time.
 
/var/log/messages:
May 26 13:39:26 LCOLLECTOR NwLogCollector[149904]: [CmdScriptCollection] [failure] [cmdscript:WrkUnit[1]payloadService:169603] [onLog:800] [azuremonitor.AzureMonitor_OWA] [processing] [WorkUnit] [processing] 2020-05-26T13:39:26Z AzuremonitorCollector Got exception in Partition Pump 1. Exception Unterminated string starting at: line 1 column 5357 (char 5356)
May 26 13:39:26 LCOLLECTOR NwLogCollector[149904]: [CmdScriptCollection] [failure] [cmdscript:WrkUnit[1]payloadService:169603] [onLog:800] [azuremonitor.AzureMonitor_OWA] [processing] [WorkUnit] [processing] 2020-05-26T13:39:26Z AzuremonitorCollector Aborting
May 26 13:39:26 LCOLLECTOR NwLogCollector[149904]: [CmdScriptCollection] [info] [cmdscript:WrkUnit[1]payloadService:169603] [onLog:794] [azuremonitor.AzureMonitor_OWA] [processing] [WorkUnit] [processing] 2020-05-26T13:39:26Z AzuremonitorCollector AzuremonitorCollector aborted.
 


 

Cause

This is due to Microsoft sends an incorrect format log to Microsoft API. Microsoft API accepts logs only in json format. When logs not coming in json format, the collection stops.


Resolution

Please download the attached azuremonitor_collector.py and follow the below steps to fix the issue.
  1. Take a backup of the existing azuremonitor_collector.py file to another location using the below command.
    mv /etc/netwitness/ng/logcollection/content/collection/cmdscript/azuremonitor/azuremonitor_collector.py /root/
     
  2. Place downloaded azuremonitor_collector.py under /etc/netwitness/ng/logcollection/content/collection/cmdscript/azuremonitor/ using WINSCP.
Note: No additional changes are required, and it should start working fine.

This plug-in will create a new event with a key called 'malformed_json' and the malformed data is added as a field to this key. This will ensure that Collector does not stop collection due to malformed data and move ahead.

Product Details

RSA Product Set: NetWitness Platform
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.2.1.0
Platform: CentOS
O/S Version: 7

Summary

This document outlines the procedure to fix Azure monitor collection crash and to persist collection.


Approval Reviewer Queue

Technical approval queue