Some RSA NetWitness hosts cannot communicate with Admin node and the status of the host and services are offline in UI
Issue
After reboot one of NetWitness appliances, the host cannot communicate with Admin node and the status of the host and services are offline in UI.
Following salt-minion errors are occurring.
Jan 4 16:38:15 DECODE1 salt-minion: [ERROR ] Error while bringing up minion for multi-master. Is master at 10.130.xx.xx responding?
Jan 4 16:39:05 DECODE1 salt-minion: [ERROR ] Error while bringing up minion for multi-master. Is master at 10.130.xx.xx responding?
In the output of 'tcpdump host 10.130.xx.xx' command, there are only [S] flag(start) of packets without any response packets. Something is blocking the communication via almost ports.
04:46:47.982104 IP 10.130.xx.xx.35279 > 10.130.xx.yy.5671: Flags [S], seq 3107709517, win 29200, options [mss 1460,sackOK,TS val 2287936306 ecr 0,nop,wscale 7], length 0
04:46:48.894344 IP 10.130.xx.xx.58717 > 10.130.xx.yy.5671: Flags [S], seq 610418755, win 29200, options [mss 1460,sackOK,TS val 2287937218 ecr 0,nop,wscale 7], length 0
04:46:49.792694 IP 10.130.xx.yy.49932 > 10.130.xx.xx.4506: Flags [S], seq 142633569, win 29200, options [mss 1460,sackOK,TS val 168378372 ecr 0,nop,wscale 7], length 0
Cause
Firewall configuration was handled by iptables service, but firewalld service is active and running after the rebooting. The config of firewalld block the communications between the host and admin node.
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-01-04 14:55:44 KST; 1h 48min ago
Docs: man:firewalld(1)
Main PID: 5520 (firewalld)
CGroup: /system.slice/firewalld.service
`-5520 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
Jan 04 14:55:43 DECODE1 systemd[1]: Starting firewalld - dynamic firewall daemon...
Jan 04 14:55:43 DECODE1 python[5520]: OWB:ERROR:RES:(crypto, MD5 (4), 0x0) not available in FIPS mode
Jan 04 14:55:43 DECODE1 python[5520]: OWB:ERROR:BSAFELIB:func(123):reason(109):b_hash.c:74
Jan 04 14:55:43 DECODE1 python[5520]: OWB:ERROR:BSAFELIB:func(122):reason(109):b_hash.c:103
Jan 04 14:55:44 DECODE1 systemd[1]: Started firewalld - dynamic firewall daemon.
Workaround
firewalld service must be stopped and disabled when using the iptables service:
Stop and disable the firewalld service.
# systemctl disable firewalld.service
Internal Comments
Gyeonghwan Hong - 7 Jan 2021Created the article.
Product Details
RSA Product Set: RSA NetWitness PlatformRSA Product/Service Type: Security Analytics Server
RSA Version/Condition: 11.3.x, 11.4.x
O/S Version: Centos7
Summary
Some RSA NetWitness hosts cannot communicate with Admin node and status of host, and services are offline in UI.
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue