NetWitness Core Services Storage sessionDB Partition Fills Up

NetWitness has identified an issue with NetWitness Platform 11.6 or newer that can cause the database storage partition to fill up instead of rolling out older data when total used database storage space reaches 95% (the expected default behavior). This can occur on any core service which includes the archiver, broker, concentrator, network decoder, packet decoder, network hybrid, packet hybrid, log retention hybrid, and endpoint log hybrid. Indicators for this issue:

1. Session db used size goes beyond 95% like 98-100%, eventually capture/aggregation stops as free space threshold hits.
2. In an environment with multiple filesystems configured as meta.dir or packet.dir or session.dir , the service writes the new DB files to the filesystem which is already filled up and service aggregation/capture  stops
3. Service stops capture or aggregation with the error “Not enough free space” in the logs

Recommendation

Fix for the storage issue is available in a 11.6.1.x and 11.7.1.x Hot Fix, as well as it will be available in upcoming 11.7.1.2 and 12.0 versions. Please reach out to your account manager or NetWitness Support for further assistance.

NetWitness would recommend to maintain monitoring of the storage partitions by using the activated hardware policy within NetWitness's Health & Wellness. This will help to notify you if a storage partition exceeds its default of 95%. If notifications have not been activated for your Health & Wellness, it is highly recommended that this is setup as soon as possible. Please see "Add an Email Notification in the Manage Policies Page

We are sorry for any inconvenience this issue may cause. If you have any questions, please reach out to your account manager or NetWitness Support.