NetWitness Hunting Guide - Page 5
NetWitness Hunting Guide - Continued
File Transfer Protocol (FTP)
The FTP protocol has been around for approximately 30 years as of the writing of this document. It is a very simple protocol and does not offer much verbosity in the sessions. One thing to note is that the STOR and RETR commands are mapped into the action metadata category as PUT and GET, respectively. Since most organizations are behind some sort of NAT boundary, passive FTP is the most common type of FTP session.
Passive vs Active FTP explanations are easily searchable on the Internet. However, the important aspect of their behavior is that, in both cases, data transfer is through another distinct TCP session. This means that the FTP service should have no forensic fingerprints available as metadata. Instead, we must go by extension and command to initially begin analyzing FTP data.
If the analyst is interested in exfil by FTP, a query of ‘service=21 && action = ‘put’ && session.analysis = ‘first carve’’ would display all of the FTP STOR commands made to IP space outside of your organization. The analyst could then pivot into filename, extension and destination organization, looking for outliers. Another interesting artifact of FTP is that there is no attempt to obfuscate the login credentials supplied by the user or application. They are simply plaintext commands of USER and PASS. By opening the username and password metadata categories the analyst can further divide up FTP traffic.
Of note is what we call ‘lazy left-handedness’. This is a password selection method when attempting to come up with a “secure” password in a hurry. For example, given a password ‘RSAhunter’, a lazy left-handed permutation of this could be ‘RSAhunter!@#’ since those special keys are under the 1, 2 and 3 key if the shift key is pressed. In the same way, ‘!qaz@wsx#edc’ might seem like a secure password, but in fact it is just running a finger down the three left most alpha key columns on a QWERTY keyboard and pressing Shift a few times. These sessions might be few and far between with the operational pace and patience shown by advanced attackers, but in more than a few cases this has been the first discovery in an investigation.
Command and Control via FTP, while rare, still does occur and should be checked for. When the C2 is up and listening, the Trojan will issue STOR for possible data about the infected machine and issue RETR for commands. With a busy attacker there should be many of these sessions. When the C2 is not listening, there should be TCP SYN beaconing to that IP that is left unanswered. It could be possible for a static username and password to be provided, or even randomly generated credentials. The Trojan could also map some of the other 60 commands of the protocol to different functions within the Trojan. Exploring the action metadata category for anything that does not look normal is recommended.
Remote Desktop Protocol (RDP) and Secure Shell (SSH)
RDP was more interesting before version 5.2+ started gaining mass adoption on the client and server side. RDP versions before this were in clear text and many indicators could be extracted (such as keyboard layout). When version 5.2+ is used, there is a certificate and key exchange that takes place and the rest of the session is encrypted. The only metadata that will be interesting in these sessions are the IP addresses and the ports, typically. Similarly, SSH leaves the analyst with little metadata to interrogate. The two protocols are usually examined separately but the methods remain the same. With this in mind we can examine our data set and look for RDP and SSH sessions.
For outbound traffic to the internet, it is generally not a good idea to allow RDP or SSH. Many employees use these to connect remotely to their home machines or machines they rent on the internet. In many cases, these sessions are not on the default port: RDP default TCP 3389 or SSH default of TCP 22. The most common non-standard ports are 80 and 443 for this traffic, as most organizations, to their detriment, allow these ports unfettered access to the internet. SSH is often used to tunnel other protocols and can act like an unauthorized VPN connection or even bridge your network to an advanced attacker.
An interesting aspect of SSH is the cleartext declaration of the SSH client and the SSH server client and server. This can allow the analyst to profile more of the behavior and also allows for a few analytical insights. If a machine is generating an SSH session to the internet and the server has a very old version of SSH, we can infer that the machine has not been patched for some time and raises the suspicion that perhaps that machine has been compromised and further investigation is needed. Similarly, the client used can help uncover the intentions of the initiator.
Similarly, encrypted RDP sessions exchange certificates to protect the private key or nonce exchange and their certificate details will populate ssl.subject and ssl.ca.
Inbound RDP and SSH should be scrutinized the most. If an internet facing server is compromised, there will most likely be attempts to RDP or SSH to that system to give the attacker rapid access to the environment in a familiar setting. Even organizations that do not allow TCP 3389 or TCP 22 connections inbound can still fall victim to this very simple method of access. RSA has observed attackers querying for and shutting down inbound services like FTP, changing the Terminal Services configuration to listen on TCP 21 and starting the service. SSH can be abused in a similar manner. There also exists metadata from Live that keeps track of Tor exit nodes and VPN service IP addresses. These should be utilized if there is too much data to efficiently examine.
A typical request asked of an analyst when discovering unauthorized outbound or inbound RDP or SSH session is to determine if anything was exfiltrated or possibly dropped. Since most RDP sessions are encrypted, and all SSH sessions are encrypted, this cannot be determined from network traffic alone except in some very specific cases. However, activity can be inferred from the network traffic. The first thing an analyst should do is select the IP addresses in question, such as in the figure below.
- Column 1:
(ip.src=192.168.1.10 || ip.dst = 192.168.1.10 || alias.ip = 192.168.1.10) && (ip.src=192.168.5.5 || ip.dst = 192.168.5.5 || alias.ip = 192.168.1.10)
Figure 26. Example IP Query
We structure the Where clause like this since the Decoder service is not actually tracking TCP handshakes and ordering the request and response accordingly. Most RDP and SSH sessions will be over 60 seconds, so this will leave plenty of continuation sessions labeled with the session.split meta. The TCP SRC port does not change during the length of the actual session, so that should be used as a guide when determining which session and continuation data to examine. Extracting the pcap in NetWitness and opening it in Wireshark will show the session with all the frames in one pcap. By navigating to the Statistics menu and then selecting endpoints, we can see received and transmitted bytes for each IP under IPv4.

Figure 27. Wireshark Endpoints Byte Counts
Internet Control Message Protocol (ICMP)
ICMP is much more than the ping command. ICMP is a standard messaging protocol that assists layer 3 and 4 protocols. ICMP is a layer 3 protocol, IP Protocol 1, with an 8 bit field for code and an 8 bit field for status. The codes most analysts are aware of are 8 and 0 which are Echo Request and Echo Response. Another popular ICMP code is 11, or Time Exceeded (TTL), often used for traceroute. ICMP accounts for a good portion of every network, yet the sessions sizes are generally small. The icmp.lua parser parses the ICMP Types and Codes from the IP Protocol 1 sessions. A Service is not registered, because ICMP is not a service, only 1 service can be registered per session and ICMP has been used to tunnel other protocols which will be natively parsed by NetWitness and their Service identified. The figures below show the common Types and Codes to appear in the network as well as the two associated alerts.
- Metadata:
large icmp request frame
- Alert Description:
Request frame is over 96 bytes
- Metadata:
large icmp response frame
- Alert Description:
Response frame is over 96 bytes

Figure 28. IR_1_ICMP.lua Example Metadata
The Session Characteristics meta large icmp request frame and large icmp response frame deserve some clarity. This logic will fire on one of the most common type codes 3 or Destination Unreachable. When this ICMP message is returned to the system that generated the traffic, it also includes a portion of the offending frame in the payload section of the frame. If it contains enough of the protocol, it will match on one of our protocol parsers. Common services found in this manner are typically Datagram-based protocols. TCP protocols need to establish a socket before sending payload data, so if a destination is unreachable there will be no payload. That being said, routing mishaps happen and TCP frames can end up inside a destination unreachable ICMP packet.
The large ICMP sessions metadata will yield many sessions. And because there are so few metadata entries for the sessions, it might seem difficult to parse through. By focusing on the session size metadata from ne Hunting Pack, the analyst is able to carve up the dataset more effectively and focus on the source IP addresses within the network that are connecting to the Internet.
If the analyst finds entries in the Forensic Fingerprint metadata category, these should be analyzed and followed up as they could indicate malicious activity in seemingly helpful ICMP sessions. Common Trojans that use ICMP try to blend in with Echo Request and Echo Reply messages, which can be selected and analyzed. If an action or error meta category is populated with an entry that has Reserved or Deprecated in it, the sessions should be analyzed for malicious behavior. These Types and Codes are not in use and could indicate a signaling method within a Trojan. Compliant operating systems should not generate these types of messages.
Common Internet File System (CIFS) Protocol
Depending on where NetWitness is capturing, the analyst will likely see very little SMB/CIFS traffic. Core traffic of a network is useful to have, but it requires a significant amount of hardware to ingest that data and then allow an analyst to search it in reasonable time frames. There is also the problem with layer 2 pathing and the standard Core -> Distribution -> Access model that most layer 2 environments share. That being said, if the organization's CIFS traffic is captured by NetWitness, there are several indicators that can be used to find lateral movement within an organization.
CIFS and RPC are used by attackers mainly for two things: transferring unnamed tasks or At jobs/Scheduled Tasks and copying files across the network. Finding At jobs is relatively easy; Live includes the parser named_pipes.lua. This Lua parser examines traffic and extracts named pipes and registers them under the metadata category named.pipe. A named pipe is a logical connection between two endpoints that is handled by CIFS. This simplifies the transport layer and also allows the action to be taken on the remote system with the privileges used when authenticating. This is taken care of by the IPC$ share or inter-process communication share. The named pipe ‘ATSVC’ signifies the At Service was invoked and an unnamed task has been sent to the remote host.
RSA has also observed lateral movement through an organization with custom binaries that use their own named pipes for communication and this parser will register their identifier as metadata, as well. Creating a feed of named pipes commonly used within an organization and alerting on new metadata is a good practice when looking for interesting pipes.
Attackers also move files around the network with CIFS. This could include exfil data, malware, tools, and webshells. Keeping this in mind can help to discover some of these artifacts, using the following queries to NetWitness.
- Example CIFS Query:
service = 445 && action = ‘create’,’write’,’read’ && fingerprint = ‘rar’
- Description:
Shows RAR files copied over CIFS
- Example CIFS Query:
service = 445 && action = ‘create’,’write’ && extension = ‘php’,’asp’,’aspx’,’jsp’,’cgi’,’pl’ && directory contains ‘iis’,’www’
- Description:
Shows web application scripts copied over CIFS, directory is not indexed by , , , , ’write’ && fingerprint contains ‘exe’
, , directory is not indexed by default, , ’write’,’read’ && directory contains ‘temp’, , , ,>Domain Name Service (DNS),>In the above sections, many of the first drills into the NetWitness dataset are made to exclude DNS. This is because when looking through the drills for HTTP, much of the analysis happens on the domain names themselves and their TLDs. Now that we pivot into DNS exclusively, we can analyze some of the aspects of this protocol.> ,>Dynamic DNS is offered on free and paid levels. Its original intent was to provide DNS services without having to register with companies such as GoDaddy or NameCheap. Many people in technology used the service to provide a way to get back to their home networks if their IP address changed on their home network. One of the aspects of Dynamic DNS is that most providers offer anonymity, meaning they don’t require information that could positively identify a person. Additionally, the TTL for most of these services is 5 minutes or 300 seconds, meaning that a new A record could reach the intended application in just over 5 minutes. This proved advantageous to an attacker and the atomicity proved annoying to the forensic analyst.> ,>The dyndns.lua parser available on Live is a growing collection of Dynamic DNS provider domains. It requires the http.lua parser and dns_verbose.lua parser. It matches on a callback to alias.host for any of our 100,000+ known Dynamic DNS domains. Search through the Dynamic DNS resolved hosts looking for your organization's name or permutations of that name, such as ‘3mc.dyndns.org’. If a suspicious domain is discovered, the resolved IP address of the DNS query, if successful> ,>There are options available that determine when the system alerts on low time-to-live (TTL) thresholds. The default values are as follows:,>function ttlLow()
--[[
"Low TTL Threshold" : default 600
Alerts for a low time-to-live value for resource records - this is the value in
seconds under which the alert will be registered. A value of 0 disables the alert.
--]]
return 600
end,>function ttlVeryLow()
--[[
"Very Low TTL Threshold" : default 60
Alerts for a very low time-to-live value for resource records - this is the value in
seconds under which the alert will be registered. A value of 0 disables the alert.
--]]
return 60
end,>To change these options, save your settings in a file named as DNS_verbose_lua_options.lua, and deploy the file to the same directory as parsers:Note the following:,> - The parser is not dependent upon the options file. The parser will load and run even in the absence of the options file. The options file is only required if you need to change the default settings.
- If you do not have an options file (or if your options file is invalid), the parser uses the default settings. ,> Note: The parser never uses both the defaults and customized options. If the options file exists and its contents can be loaded, then the defaults will not be used at all.,> ,>The alias.ip metadata category can also be used as a passive DNS system for your organization. The concentrator metadata retention rate varies by amount of content applied and data rates, but API queries can be used to extract this data and store it in another application for retrieval and long term storage.,> ,>If your organization exposes DNS servers to the internet because they are authoritative for a domain or domains, it is a good idea to inquire about the recursion policy or better yet, look for yourself. First, select the metadata you have created for inbound from the Internet traffic and select
,>If you see a domain in alias.host that does not belong to your organization that successfully resolves, you are probably hosting a recursive DNS server. This is important since the majority of DDoS activity on the Internet currently is related to open recursive DNS servers. If I were to get Internet access that does not prevent IP spoofing, I could craft ANY DNS queries with the source IP of my target and destination IP of an open recursive DNS server. The ANY query asks for all types of records like IN, MX, NS>
,>Figure 29. DNS ANY Query Illustrating Amplification Attacks,>Queries to search for this type of attack are relatively simple with the DNS verbose parser. The figure below illustrates the metadata in risk.suspicious.,>
,>Figure 30. NetWitness Metadata Indicating Many Responses,>DNS tunneling and tunneling of other protocols on the default DNS Zone Transfer port TCP/53 can be detected by examining the traffic on TCP/53 and UDP/53 where service != 53. Examining the session size and ratio transmitted meta in this drill can help sort out what you are examining and allow you to discover malicious command and control or data exfiltration via the DNS ports and protocols.>Conclusion,>Through data enrichment by developing content around specific technical aspects and behavior, an analyst can quickly find the malicious activity within terabytes of data. By actively hunting through a dataset, an informed analyst can discover new malicious activity without the need for malware specific signatures. In fact, NetWitness has been able to discover and track malicious activity using authenticated and authorized channels, which no IDS regardless of vendor or generation> ,>Finding malicious activity is not the only use case, but having a forensics repository of traffic is incredibly useful when an advanced adversary is discovered. Attacker dwell times are measured in months, if not years, and having the traffic on hand and easily exportable saves tremendous amounts of time during an incident and can also lead to many net new discoveries. Having a plan when analyzing a dataset and also being self-aware of blind spots and developing ways to cover them can reduce this dwell time and speed remediation efforts.> ,>Additionally, a tool like NetWitness can help any organization assess the state of its security posture by providing detailed visibility into its traffic. Making sure appropriate policies are in place cannot prevent an attack or breach, but it makes it harder for the breaches to be successful, and easier to detect, since ‘sessions’ that look out of place become easier to spot.> ,>BACK NEXT,