Skip to content
  • There are no suggestions because the search field is empty.

RSA NetWitness Logcollection stopped due to corrupted XFS file system

Issue

  • LogCollection Stopped. 
  • Running command: df -kh shows that the LogCollector director is fully utilized [100% full].
 
  • User-added

Cause

Corrupted "xfs" filesystem within the /var/netwitness/logcollector directory .


Resolution

  1. vi the /etc/fstab file to hash the NFS mapping of the LogCollector directory.
    User-added

    Put # prior the line highlighted with Red to be like: 
    #/dev/mapper/VolGroup00-lcol /var/netwitness/logcollector xfs     nosuid,noatime  1 2
  2. Reboot the appliance using command: reboot.
  3. Check which process is accessing that directory using command: lsof
  4. stop nwlogcollector service. 
  5. Unhash the LogCollector NFS mapping. 
  6. Unmount the directory using command: umount /dev/mapper/VolGroup00-lcol
  7. Perform  filesystem check using command: xfs_check /dev/mapper/VolGroup00-lcol
  8. Perfom filesystem repair using command: xfs_repair -L /dev/mapper/VolGroup00-lcol
  9. Re-mount the LogCollector durectory using command: umount -a /dev/mapper/VolGroup00-lcol
  10. Run command "Start nwlogcollector service" to start logcollection service. 
  11. Issue command:  df -kh to check the storage status of the "/var/netwitness/logcollector" directory you will find it dropped down around 30% to reach 70% of utilized storage only. 
  12. Run command tail -f /var/log/messages to track the operations of appliance. 
  13. Perform required health-checks by checking the health &Wellness on the SA UI.

Product Details

RSA Product Set: Security Analytics (10.5.x.x, 10.6.x.x), Netwitness for Logs and Network (11.x)
RSA Product/Service Type: Log Collector (Can be applied to any host that exhibits the same symptoms but was initially discovered on a Log Decoder/Log Collector)
RSA Version/Condition: 10.5.x.x and higher releases
Platform: CentOS 6, 7

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue