Concentrator Stuck on Re-Indexing Due to Multiple Re-Indexes in RSA NetWitness Platform

Issue

In the UI, the Concentrator config page is stuck with a status of indexing.
This may be due to multiple reindex instances.
While looking at the Concentrator /var/log/messages logs, similar messages below are displayed:

 [Index] [failure] Reindex operation created 2, expected 1 
 
[Index] [failure] Throw in function bool nw::MetaIndexEngine::indexLastGap(bool)Dynamic exception type: boost::exception_detail::clone_impl<nw::Exception>std::exception::what: Reindex operation created 2, expected 1[boost::errinfo_at_line_*] = 696
 
[Aggregation] [info] Device 'xxxx' is attempting to restart aggregation from failed state. 
 
[Aggregation] [info] Device 'xxxx' is being initialized 
 
[Aggregation] [failure] Failed to find last session device 'xxxx' because waiting on indexing. Device aggregation is being stopped

Cause

The re-index process stopped due to the re-indexer finding multiple slices while expecting 1 slice. An additional index save was triggered which created the extra slice.

Resolution

Slices that are located in /var/netwitness/concentrator/index/reindex need to be moved in to /var/netwitness/concentrator/index.
Once completed, restart the concentrator service

For CentOS 6.x-based appliances

Restart nwconcentrator

For CentOS 7.x-based appliances

systemctl restart nwconcentrator

If you are unsure of any of the steps above or experience any issues, contact RSA Support and quote this article ID for further assistance.

Product Details

RSA Product Set: NetWitness Platform, Security Analytics
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.x,10.6.x
Platform: CentOS
O/S Version: EL7, EL6

Summary

Concentrator not consuming from downstream decoders and config page showing re-indexing

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue