NetWitness Unable to Query on the Broker from Investigate

Issue

Performing a custom drill on a Broker yields no results, while the same custom drill on the downstream Concentrator displays the requested meta.
The NetWitness Broker is consuming normally from the Concentrator with no sessions behind, but does not yield the same results during an investigation.
All meta keys in the Investigation UI display the following message when querying the broker:

 Meta not available on device

The Investigation UI timeline displays an error similar to the following when querying the broker:

 Failed to retrieve counts for sessions, sizes, and packets across a time range Dec 31, 1969 to Jan 1, 1970: 400 Bad Request

Cause

This issue can be a symptom of the /var/netwitness/broker partition becoming 100% utilized due to a buildup of core dump files.

Resolution

After confirming that none of the core dump files need to be preserved for further analysis, the core dump files may be removed by issuing the command below.

rm /var/netwitness/broker/core*

After removing the files and the utilization of the partition has been reduced, the symptoms will no longer occur and investigations may be performed normally against the broker appliance.

If you are unsure whether or not this case applies to your case or if you experience any issues, please contact our support: Support Phone Numbers

Notes

Code block of the df -h command, showing that the /var/netwitness/ partition is 100% utilized.

 [root@Broker ~]# df -h
 
Filesystem Size Used Avail Use% Mounted on
 
devtmpfs 32G 0 32G 0% /dev
 
tmpfs 32G 28K 32G 1% /dev/shm
 
tmpfs 32G 153M 32G 1% /run
 
tmpfs 32G 0 32G 0% /sys/fs/cgroup
 
/dev/mapper/netwitness_vg00-root 30G 9.7G 21G 33% /
 
/dev/mapper/netwitness_vg00-varlog 10G 4.0G 6.1G 40% /var/log
 
/dev/sda1 1014M 118M 897M 12% /boot
 
/dev/mapper/netwitness_vg00-nwhome 141G 141G 496M 100% /var/netwitness
 
/dev/mapper/netwitness_vg00-usrhome 10G 33M 10G 1% /home
 
tmpfs 6.3G 0 6.3G 0% /run/user/0

Code block of the ls -lah /var/netwitness/broker command, showing the core dump directory that is present in the directory.

 [root@Broker ~]# ls -lah /var/netwitness/broker
 
total 8.0K
 
drwxr-x---. 5 netwitness netwitness 46 Jan 12 18:18 .
 
drwxr-xr-x. 35 root root 4.0K Apr 2 19:03 ..
 
drwxr-xr-x. 3 root root 21 Jan 21 2023 cache
 
drwxr-xr-x. 2 root root 28 Jan 21 2023 cores
 
drwxr-xr-x. 2 root root 28 Jan 21 2023 index
 
drwxr-xr-x. 2 root root 4.0K Apr 2 19:15 statdb

Product Details

NetWitness Product Set: NetWitness Platform
NetWitness Product/Service Type: Log Collector
NetWitness Version/Condition: 11.x, 12.x
Platform: CentOS / AlmaLinux
O/S Version: 7 / 8.9

INTERNAL ONLY!!!

Approval Reviewer Queue

Technical approval queue