Skip to content
  • There are no suggestions because the search field is empty.

The /tmp partition on an RSA  NetWitness Platform appliance is 100% utilized but no large files are present

Issue

The /tmp partition on an RSA Security Analytics appliance is 100% utilized but no large files are present.

Issuing the df -h command on the appliance shows the /tmp partition as being full, as seen in the example below.

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-root
                       20G  2.1G   17G  12% /
/dev/mapper/VolGroup00-tmp
                       20G  20G      0 100% /tmp
/dev/mapper/VolGroup00-usrhome
                      4.9G  139M  4.5G   3% /home
/dev/mapper/VolGroup00-var
                       20G  4.3G   15G  23% /var
/dev/mapper/VolGroup00-nwhome
                       62G  890M   58G   2% /var/netwitness
/dev/sdc1             251M   18M  221M   8% /boot
tmpfs                  48G     0   48G   0% /dev/shm
/dev/mapper/decodersmall-decoroot
                      9.9G  4.3G  5.1G  46% /var/netwitness/decoder
/dev/mapper/decodersmall-index
                       10G  1.1G  9.0G  11% /var/netwitness/decoder/index
/dev/mapper/decodersmall-sessiondb
                      600G  133G  468G  23% /var/netwitness/decoder/sessiondb
/dev/mapper/decoder-packetdb
                       19T   18T  945G  95% /var/netwitness/decoder/packetdb


Examining the /tmp partition with the ls -lah /tmp command shows no large files that account for the utilized disk space.

Cause

While the /tmp partition is used to store physical files, it is also used to store "virtual files," meaning files that are in use by an active process.  If a process doesn't release the file correctly, even though the file is not is not actually on the filesystem structure, the space it used is still allocated.


Resolution

In order to confirm that the issue is caused by allocated space from files not properly released from processes, you may issue the following command:  lsof | grep /tmp | grep deleted

Issuing the command above will display a list of the files that have since been deleted but are still associated with an active process and claiming disk space.  You will also be able to see the amount of space that is being consumed.(See the screenshot in the section below)

After identifying the process (or processes) that is still linked to the files, you will be able to perform one of the following three actions to free the space:

  • Restart the service/daemon that is responsible.
  • Kill the associated process.
  • Reboot the appliance.

If you are unsure of any of the steps above or experience any issues, contact RSA Support and quote this article ID for further assistance.


Notes

The screenshot below is an example of output generated by the lsof | grep /tmp | grep deleted command that was issued on a Security Analytics server appliance.

The /tmp partition on an RSA  NetWitness Platform appliance is 100% utilized but no large files are present


Product Details

RSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: NetWitness Core
RSA Version/Condition: 10.6.x, 11.x
Platform: CentOS

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue