Rabbitmq crashing due to hostname lookup failed errors in RSA Netwitness

Issue

Rabbitmq crashing with below errors on any of the core services. 

tailf /var/log/rabbitmq/rabbit.xxx.log
2022-10-18 12:54:29.190 [info] <0.686.0> Resetting node maintenance status
2022-10-18 12:54:31.401 [error] <0.912.0> Channel error on connection <0.873.0> (10.70.189.32:54483 -> 10.70.189.34:5671, vhost: '/rsa/system', user: 'e8141cdd-d206-470d-9e5c-6db056ac167b'), channel 3:
operation exchange.declare caused a channel exception not_found: no exchange 'carlos.sms.collectd' in vhost '/rsa/system'


tailf /var/log/messages 
Oct 18 11:52:01 MOLBUDARSAARCH01 rabbitmq-server: Logger - error: {removed_failing_handler,rabbit_log}
Oct 18 11:52:18 MOLBUDARSAARCH01 NwArchiver[1707]: [AMQP] [failure] Error setting client certificate for socket or timeout waiting for response: hostname lookup failed

Cause

This issue occured to rabbitmq is pointing to wrong admin-node in its configuration. This might have occured during host migration activity.

Resolution

Perform these steps on any of the node-x servers:-

1. Please have a look at below configuration file. For Instance Archiver in this article.
#cat /etc/netwitness/ng/NwArchiver.cfg | grep -i amqp
  
2. Modify above highlighted lines to have correct admin-server IP replacing x.x.x.x and its associated node ID. 
3. Restart rabbitmq 
#systemctl restart rabbitmq-server

Product Details

RSA Product Set: RSA NetWitness Platform
RSA Product/Service Type: Security Analytics Server
RSA Version/Condition: 11.6 and Above
Platform: CentOS
O/S Version: 7

Approval Reviewer Queue

Technical approval queue