Error: Failed to start RabbitMQ broker in RSA NetWitness 11.x

Issue

Rabbitmq service fails to start.

 [root@DECODER ~]# systemctl start rabbitmq-server.service
 
Job for rabbitmq-server.service failed because the control process exited with error code. See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.

 [root@DECODER ~]# journalctl -xe
 
........
 
-- Unit rabbitmq-server.service has begun starting up.
 
Aug 19 01:46:45 tspl-conc rabbitmq-server[2427]: ERROR: epmd error for host 3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5: nxdomain (non-existing domain)
 
Aug 19 01:46:45 tspl-conc systemd[1]: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
 
Aug 19 01:46:46 tspl-conc rabbitmqctl[2548]: Stopping and halting node 'rabbit@3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5'
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: Error: unable to connect to node 'rabbit@3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5': nodedown
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: DIAGNOSTICS
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: ===========
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: attempted to contact: ['rabbit@3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5']
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: rabbit@3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5:
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: * unable to connect to epmd (port 4369) on 3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5: nxdomain (non-existing domain)
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: current node details:
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: - node name: 'rabbitmq-cli-70@DECODER'
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: - home dir: /var/lib/rabbitmq
 
Aug 19 01:46:47 tspl-conc rabbitmqctl[2548]: - cookie hash: hZaV/dJZ6+9JXEWxkAJdjg==
 
Aug 19 01:46:47 tspl-conc systemd[1]: Failed to start RabbitMQ broker.

Cause

The reasons causing this problem would vary depending on the situation.
This KB just describes one of the reasons why Rabbitmq service fails to start in Netwitness 11.x.
In this case, salt minion id is missing in /etc/hosts file.

 [root@DECODER ~]# cat /etc/hosts
 
# Created by NetWitness Installer on Thu Jul 18 00:29:38 UTC 2019
 
127.0.0.1 DECODER localhost.localdom localhost

Resolution

Update the /etc/hosts with the salt minion id and start rabbitmq service.

 [root@DECODER ~]# cat /etc/salt/minion
 
master: x.x.x.x
 
hash_type: sha256
 
log_level: info
 
id: 
 3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5

 [root@DECODER ~]# cat /etc/hosts
 
# Created by NetWitness Installer on Thu Jul 18 00:29:38 UTC 2019
 
127.0.0.1 DECODER localhost.localdom localhost 
 3a0569cb-7eb0-4d7e-bc1e-5abfcf6e82e5

Product Details

RSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.x
Platform: CentOS 7

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue