Cannot Connect to System Management.service Error in UI frequently
Issue
Netwitness UI frequently gets Error: "Cannot Connect to System Management.service" as below.
Checking SMS service status using below command shows service running.
systemctl status rsa-sms.service
/opt/rsa/sms/wrapper.log:
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "pool-86-thread-1" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS esmStat Parser Worker Thread 878075291" Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS esmStat Parser Worker Thread 878075291" Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2020/10/04 15:42:43 | Exception in thread "SMS smsStat Parser Worker Thread 1539714682" java.lang.OutOfMemoryError: Java heap space
Cause
This issue is due to sms running out of JVM heap memory.
Workaround
Restarting SMS service using below command brings all stats back in Health & wellness page without error.systemctl restart rsa-sms.service
Resolution
Please follow the below steps to fix the error permanently.- Login putty of NwServer and Stop SMS service using the below command.
systemctl stop rsa-sms.service - Edit the file /opt/rsa/sms/conf/wrapper.conf and add following line.
wrapper.java.additional.1=-Xmx32g
Note: By default 16GB heap space allocated to SMS service. This allocation of heap space can be increased to high if system has some overhead memory available.
- Start SMS service using the below command.
systemctl start rsa-sms.service
- SMS status command shows the 32GB heap memory as below.
#systemctl status rsa-sms.service● rsa-sms.service - RSA NetWitness SMS :: Server
Loaded: loaded (/usr/lib/systemd/system/rsa-sms.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2020-10-28 11:17:18 UTC; 1 weeks 6 days ago
Main PID: 105637 (wrapper-linux-x)
CGroup: /system.slice/rsa-sms.service
├─ 85359 java - Xmx32g -XX:+UseG1GC -Djavax.net.ssl.keyStore=/opt/rsa/sms/
Product Details
RSA Product Set: RSA NetWitness PlatformRSA Product/Service Type: Reporting Engine
RSA Version/Condition: 11.3.2.0
Platform: CentOS
O/S Version: 7
Summary
This document outlines the procedure to fix error permanently.
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue