I have been working with a few customers to add custom CEF log sources to SA and got into using Lua to parse logs instead of customizing the cef.xml parser or other default parsers. VxStream logs came my way via a side project from the developers of the sandbox software from Payload Security.
https://www.payload-security.com/products/vxstream-sandbox
If you are looking for an alternative sandbox this one looks pretty interesting with a huge number of behaviour detections to flag and create reports. I haven't focused on getting the files from either packets or malware to VxStream Sandbox yet but we were assured that there is an API that can be leveraged to post files to the sandbox.
Back to CEF logs... how would be onboard these CEF formatted logs to RSA NetWitness logs without customizing the default cef.xml parser ? CEF by default will parse the items in the first part of the message where the | values are. Once you get past that, the cn* and cs* will need to be extracted manually with Lua.
Here is a sample log from Payload Security. (default extractions in bold):
Aug 18 10:26:15 aaa.xxx.t-yyyy.de CEF:0|Payload Security|VxStream|5.00|Sample Analysis Result - Malicious|Sample Analysis Result - Malicious|100|end=08/18/2016 15:22:05 cn1=100 cn1Label=Threat Score cn2=62 cn2Label=AV Detection Rate cs1=Trojan.GenericKD cs1Label=Malware Family cs2=4 cs2Label=EnvironmentID cs3=W7 32 bit Kernelmode cs3Label=Environment Description fileHash=8d79bba763f5cbe4b778ddae6de1c97a9aca7049763466ffc289cf1306c71932 fname=Multi_Process.bin fsize=2474496 fileType=PE32 executable (GUI) Intel 80386, for MS Windows request=https://www.hybrid-analysis.com/sample/8d79bba763f5cbe4b778ddae6de1c97a9aca7049763466ffc289cf1306c71932/?environmentId\=4 msg=Malicious flexString1= flexString1Label=Uploader Comment cs4=degreat247.no-ip.biz \ndegreat248.no-ip.org cs4Label=Contacted Domains cs5=69.62.234.85:7070 cs5Label=Contacted Hosts cs6= cs6Label=Compromised Hosts cs8=2013743 \n2013743 \n2013743 \n2013743 \n2013743 \n2013743 \n2013743 \n2013743 \n2013743 \n2013743 cs8Label=ET Alerts priority=9
From this we are going to extract meta from the CEF format where the default data isn't extracted (cs or cn fields):
· Device IP -> IP of the VxStream Sandbox sending the logs
· Medium -> 32 is RSA NW internal for logs (packets is 1)
· Device.type -> payload_security_vxstream (Payload Security|VxStream)
· Event.time.str -> analysis start time (08/18/2016 15:22:05)
· Alias.host -> domain name of the VxStream service/appliance (aaa.xxx.t-yyyy.de)
· Product-> VxStream (VxStream)
· Version -> version of the VxStream service/appliance (5.00)
· Event.type -> from the CEF message (Sample Analysis Result – Malicious)
· Event.desc -> from the CEF message (Sample Analysis Result – Malicious)
· Severity -> from the CEF message (100)
· Checksum -> filehash (8d79bba763f5cbe4b778ddae6de1c97a9aca7049763466ffc289cf1306c719320)
· Filename -> fname (Multi_Process.bin)
· Extension -> from the filename (.bin)
· Filename.size -> fsize (2474496)
· url -> request (https://www.hybrid-analysis.com/sample/8d79bba763f5cbe4b778ddae6de1c97a9aca7049763466ffc289cf1306c71932/?environmentId\=4 msg=Malicious flexString1= flexString1Label=Uploader Comment)
· virusname -> cs1 (Trojan.GenericKD)
· risk.num.sand -> cn1 (75)
o Above 90 – very sure
o 75 – pretty sure
· Event.type -> Sample Analysis Result – Malicious ( Malicious, Suspicious, No Threat, Unknown)– matches with RSA Sandbox malware meta
To Do (requires more Lua foo)
· alias.host -> cs4 (cs4=degreat247.no-ip.biz \ndegreat248.no-ip.org cs4Label=Contacted Domains
· ip.dst -> cs6 (cs6=52.58.99.202 \n38.229.70.4 \n217.197.83.197 \n93.184.220.29 \n52.85.184.221 cs6Label=Compromised Hosts)
To get these fields to be indexed you need to add the following changes to the index:
Index-concentrator-custom.xml
Restart the services to bring those keys online
You might want to create meta profile to help you locate the logs and set the metagroup for you automatically.
now the parser will show up top right in the Parsers section