There has been a lot of great information published about Sysmon since Mark Russinovich's presentation at RSA Conference. Eric Partington posted a great blog showing how to use Sysmon data with RSA NetWitness for Logs: https://community.rsa.com/community/products/netwitness/blog/2017/02/28/log-sysmon-6-windows-event-collection. This prompted RSA’s IR Team to publish details on how to get the rich tracking information generated by RSA NetWitness Endpoint that they use everyday for their incident investigations into a SIEM Here.
The aim of this blog is to show you how to collect this tracking data from RSA NetWitness Endpoint with RSA NetWitness for Logs. The collection is done via the Log Collector using a custom ODBC typespec.
*** DISCLAIMER - this is a field developed Proof of Concept, shared with the Community. It is not endorsed by RSA Engineering. The database structure used by NWE may change at any time. No testing has been done to measure the impact on performance for a production NWE Server. This has been developed and tested using RSA NetWitness Endpoint v4.3.0.1 and RSA NetWitness for Logs v10.6.2.1. /DISCLAIMER ***
***DISCLAIMER 2 - for this Proof of Concept, we have disabled the requirement on the NWE SQL Server to Force Encryption. /DISCLAIMER 2 ***
The objective of this integration is to get the tracking data from NWE as it is being collected into NWL, so we can index it and use it for Investigations. Tracking data in NWE can only be viewed on a per machine basis - this integration allows us to get a global view of tracking data across all of our endpoints. Here's the high level summary of what we need to do (if you want to skip to the end, all files are attached as a zip):
- Create a new ODBC typespec definition (XML file) to query the NWE Database and get the data we want,
- Create a new Log Parser to map the results of the SQL query into metadata,
- Add the meta we are using to the table-map-custom.xml so it is persistent,
- Add the meta we want to index to the index-concentrator-custom.xml file,
- Configure a new ODBC DSN definition,
- Configure a new ODBC Event Collector,
- Configure a new Meta Group to show our data for investigations,
- Configure a new Column Group to show the data we want in Events view,
- Configure some Report Rules and Charts to visualise the data,
- Configure a new RSA NetWitness Endpoint Dashboard to keep track of our environment.
Here we go!
1. Create ODBC Definition
Thanks to Andreas Funk and his blog https://community.rsa.com/community/products/netwitness/blog/2016/10/18/integrating-a-mysql-community-database-with-netwitness-for-logs for giving us a primer on how to create a new ODBC connection. We need to create a new Filespec to tell the ODBC collector how to query the NWE database and get the data we want.
On the Log Collector (either the one on the Log Decoder, or a separate VLC - whichever you are going to use to collect these logs) the ODBC collection definitions are stored here:
/etc/netwitness/ng/logcollection/content/collection/odbc/
We need to add a new file for our NWE tracking data -
vi /etc/netwitness/ng/logcollection/content/collection/odbc/nwe_tracking.xml
Here is the query from Rui Ataide's blog, modified to work for NWL, included in our definition:
nwe_tracking
odbc
NetWitness Endpoint Tracking
2.0
Chris Thomas
Import NWE Tracking data
nwe_tracking
nwe_tracking
||
30
(SELECT
SE.PK_WinTrackingEvents,
SE.EventUTCTIme,
MA.MacAddress as src_mac,
MA.LocalIp as src_ip,
MA.MachineName,
LOWER(PA.Path),
LOWER(FN.FileName),
LOWER(PA.Path + FN.FileName) AS Source,
MO.HashSHA256,
LA.LaunchArguments AS SLA,
CASE
WHEN SE.BehaviorFileOpenPhysicalDrive = 1 THEN 'OpenPhysicalDrive'
WHEN SE.BehaviorFileReadDocument = 1 THEN 'ReadDocument'
WHEN SE.BehaviorFileWriteExecutable = 1 THEN 'WriteExecutable'
WHEN SE.BehaviorFileRenameToExecutable = 1 THEN 'RenameExecutable'
WHEN SE.BehaviorProcessCreateProcess = 1 THEN 'CreateProcess'
WHEN SE.BehaviorProcessCreateRemoteThread = 1 THEN 'CreateRemoteThread'
WHEN SE.BehaviorProcessOpenOSProcess = 1 THEN 'OpenOSProcess'
WHEN SE.BehaviorProcessOpenProcess = 1 THEN 'OpenProcess'
WHEN SE.BehaviorFileSelfDeleteExecutable = 1 THEN 'SelfDelete'
WHEN SE.BehaviorFileDeleteExecutable = 1 THEN 'DeleteExecutable'
WHEN SE.BehaviorRegistryModifyBadCertificateWarningSetting = 1 THEN 'ModifyBadCertificateWarningSetting'
WHEN SE.BehaviorRegistryModifyFirewallPolicy = 1 THEN 'ModifyFirewallPolicy'
WHEN SE.BehaviorRegistryModifyInternetZoneSettings = 1 THEN 'ModifyInternetZoneSettings'
WHEN SE.BehaviorRegistryModifyIntranetZoneBrowsingNotificationSetting = 1 THEN 'ModifyIntranetZoneBrowsingNotificationSetting'
WHEN SE.BehaviorRegistryModifyLUASetting = 1 THEN 'ModifyLUASetting'
WHEN SE.BehaviorRegistryModifyRegistryEditorSetting = 1 THEN 'ModifyRegistryEditorSetting'
WHEN SE.BehaviorRegistryModifyRunKey = 1 THEN 'ModifyRunKey '
WHEN SE.BehaviorRegistryModifySecurityCenterConfiguration = 1 THEN 'ModifySecurityCenterConfiguration'
WHEN SE.BehaviorRegistryModifyServicesImagePath = 1 THEN 'ModifyServicesImagePath'
WHEN SE.BehaviorRegistryModifyTaskManagerSetting = 1 THEN 'ModifyTaskManagerSetting'
WHEN SE.BehaviorRegistryModifyWindowsSystemPolicy = 1 THEN 'ModifyWindowsSystemPolicy'
WHEN SE.BehaviorRegistryModifyZoneCrossingWarningSetting = 1 THEN 'ModifyZoneCrossingWarningSetting'
END AS Action,
LOWER(SE.Path_Target),
LOWER(SE.FileName_Target),
LOWER(SE.Path_Target + SE.FileName_Target) AS Destination,
SE.LaunchArguments_Target AS TLA,
se.HashSHA256_Target
FROM
dbo.WinTrackingEvents_P1 AS SE WITH(NOLOCK)
INNER JOIN dbo.Machines AS MA WITH(NOLOCK) ON MA.PK_Machines = SE.FK_Machines
INNER JOIN dbo.MachineModulePaths AS MP WITH(NOLOCK) ON MP.PK_MachineModulePaths = SE.FK_MachineModulePaths
INNER JOIN dbo.Modules AS MO WITH(NOLOCK) ON MO.PK_Modules = MP.FK_Modules
INNER JOIN dbo.FileNames AS FN WITH(NOLOCK) ON FN.PK_FileNames = MP.FK_FileNames
INNER JOIN dbo.Paths AS PA WITH(NOLOCK) ON PA.PK_Paths = MP.FK_Paths
INNER JOIN dbo.LaunchArguments AS LA WITH(NOLOCK) ON LA.PK_LaunchArguments = SE.FK_LaunchArguments__SourceCommandLine
WHERE PK_WinTrackingEvents > '%TRACKING%'
UNION
SELECT
SE.PK_WinTrackingEvents,
SE.EventUTCTIme,
MA.MacAddress as src_mac,
MA.LocalIp as src_ip,
MA.MachineName,
LOWER(PA.Path),
LOWER(FN.FileName),
LOWER(PA.Path + FN.FileName) AS Source,
MO.HashSHA256,
LA.LaunchArguments AS SLA,
CASE
WHEN SE.BehaviorFileOpenPhysicalDrive = 1 THEN 'OpenPhysicalDrive'
WHEN SE.BehaviorFileReadDocument = 1 THEN 'ReadDocument'
WHEN SE.BehaviorFileWriteExecutable = 1 THEN 'WriteExecutable'
WHEN SE.BehaviorFileRenameToExecutable = 1 THEN 'RenameExecutable'
WHEN SE.BehaviorProcessCreateProcess = 1 THEN 'CreateProcess'
WHEN SE.BehaviorProcessCreateRemoteThread = 1 THEN 'CreateRemoteThread'
WHEN SE.BehaviorProcessOpenOSProcess = 1 THEN 'OpenOSProcess'
WHEN SE.BehaviorProcessOpenProcess = 1 THEN 'OpenProcess'
WHEN SE.BehaviorFileSelfDeleteExecutable = 1 THEN 'SelfDelete'
WHEN SE.BehaviorFileDeleteExecutable = 1 THEN 'DeleteExecutable'
WHEN SE.BehaviorRegistryModifyBadCertificateWarningSetting = 1 THEN 'ModifyBadCertificateWarningSetting'
WHEN SE.BehaviorRegistryModifyFirewallPolicy = 1 THEN 'ModifyFirewallPolicy'
WHEN SE.BehaviorRegistryModifyInternetZoneSettings = 1 THEN 'ModifyInternetZoneSettings'
WHEN SE.BehaviorRegistryModifyIntranetZoneBrowsingNotificationSetting = 1 THEN 'ModifyIntranetZoneBrowsingNotificationSetting'
WHEN SE.BehaviorRegistryModifyLUASetting = 1 THEN 'ModifyLUASetting'
WHEN SE.BehaviorRegistryModifyRegistryEditorSetting = 1 THEN 'ModifyRegistryEditorSetting'
WHEN SE.BehaviorRegistryModifyRunKey = 1 THEN 'ModifyRunKey '
WHEN SE.BehaviorRegistryModifySecurityCenterConfiguration = 1 THEN 'ModifySecurityCenterConfiguration'
WHEN SE.BehaviorRegistryModifyServicesImagePath = 1 THEN 'ModifyServicesImagePath'
WHEN SE.BehaviorRegistryModifyTaskManagerSetting = 1 THEN 'ModifyTaskManagerSetting'
WHEN SE.BehaviorRegistryModifyWindowsSystemPolicy = 1 THEN 'ModifyWindowsSystemPolicy'
WHEN SE.BehaviorRegistryModifyZoneCrossingWarningSetting = 1 THEN 'ModifyZoneCrossingWarningSetting'
END AS Action,
LOWER(SE.Path_Target),
LOWER(SE.FileName_Target),
LOWER(SE.Path_Target + SE.FileName_Target) AS Destination,
SE.LaunchArguments_Target AS TLA,
se.HashSHA256_Target
FROM
dbo.WinTrackingEvents_P0 AS SE WITH(NOLOCK)
INNER JOIN dbo.Machines AS MA WITH(NOLOCK) ON MA.PK_Machines = SE.FK_Machines
INNER JOIN dbo.MachineModulePaths AS MP WITH(NOLOCK) ON MP.PK_MachineModulePaths = SE.FK_MachineModulePaths
INNER JOIN dbo.Modules AS MO WITH(NOLOCK) ON MO.PK_Modules = MP.FK_Modules
INNER JOIN dbo.FileNames AS FN WITH(NOLOCK) ON FN.PK_FileNames = MP.FK_FileNames
INNER JOIN dbo.Paths AS PA WITH(NOLOCK) ON PA.PK_Paths = MP.FK_Paths
INNER JOIN dbo.LaunchArguments AS LA WITH(NOLOCK) ON LA.PK_LaunchArguments = SE.FK_LaunchArguments__SourceCommandLine
WHERE PK_WinTrackingEvents > '%TRACKING%' )
ORDER By SE.PK_WinTrackingEvents ASC
PK_WinTrackingEvents
SELECT MAX(PK_WinTrackingEvents) FROM dbo.WinTrackingEvents_P0
This creates a log entry with a static format, that is delimited by a double pipe ||:
This makes it easy for us to create a new log parser.
2. Create a new Log Parser
For information on how to create a new log parser using the new Log Parser Tool, head over here: https://community.rsa.com/community/products/netwitness/blog/2017/03/08/rsa-netwitness-esi-tool-10-beta-2. We need to create a new directory where the Log Decoder parsers are kept, and add our ini and xml parser files
mkdir /etc/netwitness/ng/envision/etc/devices/nwe_tracking/
Here is the ini file that describes our parser: nwe_tracking.ini
DatabaseName=nwe_tracking
DisplayName=NetWitness Endpoint Tracking
DeviceGroup=
DeviceType=7104
And here is the Log Parser: v20_nwe_trackingmsg.xml - the meta keys to use were chosen to line up with where the data from sysmon gets mapped to, as shown here: https://community.rsa.com/community/products/netwitness/blog/2017/02/28/log-sysmon-6-windows-event-collection
name="nwe_tracking"
displayname="NetWitness Endpoint Tracking"
group=""
type="7104">
xml="1"
revision="1"
device="2.0"/>
id1="HDR1"
id2="HDR1"
messageid="STRCAT('NWEPMSG')"
content="%nwe_tracking:<trans_id>||<event_time>||<!payload:trans_id>"/>
id1="NWEPMSG"
id2="NWEPMSG"
eventcategory="1612000000" content="<trans_id>||<event_time>||<smacaddr>||<saddr>||<event_computer>||<directory>||<filename>||<parent_process>||<checksum>||<parent_params>||<category>||<directory>||<filename>||<process>||<params>||<checksum>"/>
There should be 2 files in the new directory:
[root@RSAANZSCSA nwe_tracking]# pwd
/etc/netwitness/ng/envision/etc/devices/nwe_tracking
[root@RSAANZSCSA nwe_tracking]# ls -l
total 8
-rw-r--r--. 1 root root 96 Mar 9 10:01 nwe_tracking.ini
-rw-r--r--. 1 root root 761 Mar 10 02:59 v20_nwe_trackingmsg.xml
[root@RSAANZSCSA nwe_tracking]#
3. Add meta to table-map-custom.xml
This step can be done using the Web GUI, but since we're already on the command line we'll do it there. It's always a good idea to make a back up copy of the file first!
cp /etc/netwitness/ng/envision/etc/table-map-custom.xml /etc/netwitness/ng/envision/etc/table-map-custom.xml.old
Then edit the table-map-custom.xml file:
vi /etc/netwitness/ng/envision/etc/table-map-custom.xml
We can add the meta we are using (that is not already set as persistent (flags="None") at the end of the file: