How to Remove Old Data from a Warehouse in RSA Security Analytics 10.x

Issue

If your warehouse becomes full, then many notifications will be generated such as: 

• Notifications that a warehouse connector stream has stopped processing.
• Notifications that the warehouse is full or becoming full.

These can adversely affect performance, so it is important that space on the warehouse is managed so that it does not become completely filled up.

Resolution

In order to remove old files from the warehouse create a script on a warehouse connector that will run on an hourly basis as a cron job.

Here the mount point of the NFS Share on the warehouse in the example below is /mnt/saw/lonsaw
Your mount point will be different but but will contain the rsasoc/v1/... directory structure.


The contents of the script is as follows:  

1. Create a file with the contents of the script as above (eg call it /root/saw-cleanup.sh)
2. Modify the retention time to suit your environment. For example the script above deletes files that are older than 365 days and removes any empty directories.
3. Make sure that this file is executable (chmod +x saw-cleanup.sh)
4. Run this file as a cronjob. To do this add the line below to the /etc/crontab file on the warehouse connector appliance.

This will run the script every hour at 13 minutes past the hour and delete any files older than 365 days.

Notes

Notes: The SAW hardware platform is long past EOL, however customers may still use the WHC pointed to their own warehouse and this might be helpful.

Product Details

NetWitness Product Set:  NetWitness Logs & Network
NetWitness Product/Service Type: Warehouse Connector, SAW
NetWitness Version/Condition: 10.x ,11.x, 12.x
Platform: CentOS, AlmaLinux
 

Summary

This article explains how to use a cron job to remove old data from your warehouse.

Approval Reviewer Queue

Technical approval queue