RSA NetWitness Respond Service is down due to Correlation Server IP is not reachable.

Issue

Cause

This error is due to the Correlation server IP is not reachable by the Respond service to connect mongo database.

Resolution

1. Log into NwServer (Node0) putty to check Primary Correlation server ip configured to be used by Respond service.
   [root@SA ]# security-cli-client --get-config-prop --prop-hierarchy nw --prop-name rsa.data.application.servers[0]
   2020-02-07 09:28:29.286 INFO 33930 --- [ main] Bootstrap : Service logs will be written to /var/log/netwitness/security-client
   2020-02-07 09:28:29.292 INFO 33930 --- [ main] Bootstrap : Service configuration will be read from /etc/netwitness/security-client
   2020-02-07 09:28:29.389 INFO 33930 --- [ main] Bootstrap : Starting security-client.dbedfafb-ca48-4577-94e3-cd4816c0b93f (v0.0.0.0)
   2020-02-07 09:28:29.593 INFO 33930 --- [ main] Bootstrap : Initialized service cryptography with 4 providers (BSAFE=CRYPTOJ 6.2.4.0.1 20180724 0958, FIPS-140=true).
   2020-02-07 09:28:30.342 INFO 33930 --- [ main] c.r.n.i.s.client.SecurityApplication : Starting SecurityApplication on SA with PID 33930 (/usr/bin/security-cli-client.jar started by root in /var/log/netwitness/respond-server)
   2020-02-07 09:28:30.343 INFO 33930 --- [ main] c.r.n.i.s.client.SecurityApplication : The following profiles are active: amqp
   2020-02-07 09:28:30.418 INFO 33930 --- [ main] Bootstrap : Service will accept AMQP requests at broker localhost:5672/rsa/system
   2020-02-07 09:28:30.419 INFO 33930 --- [ main] Bootstrap : Service will use the deployment security-server
   2020-02-07 09:28:31.816 WARN 33930 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.LocalDateTime to class java.time.Instant as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:28:31.816 WARN 33930 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.Instant to class java.time.LocalDateTime as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:28:31.863 WARN 33930 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.LocalDateTime to class java.time.Instant as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:28:31.864 WARN 33930 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.Instant to class java.time.LocalDateTime as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:28:32.092 INFO 33930 --- [ main] c.r.n.i.s.client.SecurityApplication : Started SecurityApplication in 3.646 seconds (JVM running for 4.119)
   2020-02-07 09:28:32.565 INFO 33930 --- [shake Completed] Security : Accepted new connection with CN=20b26165-dd2d-4b9d-8189-4d8b2956347b,OU=NetWitness Platform,O=RSA,L=Reston,ST=VA,C=US from 20b26165-dd2d-4b9d-8189-4d8b2956347b using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
   2020-02-07 09:28:32.731 INFO 33930 --- [ main] c.r.n.i.security.client.SecurityClient : Retrieving value from Config-Server for property: nw:rsa.data.application.servers[0]
   10.10.10.1
   2020-02-07 09:28:32.806 INFO 33930 --- [ main] SystemOperation : Update current versions on disk {com.rsa.asoc.compass.config-api=5.14.0, com.rsa.asoc.compass.orchestration-api=5.12.0, com.rsa.asoc.compass.security-api=5.13.0}
   2020-02-07 09:28:35.070 INFO 33930 --- [ main] c.r.n.i.s.client.SecurityApplication : Tasks completed successfully…
   [2020-02-07T09:28:35+00:00] <33918> (INFO) Request completed successfully.
   
   
    
2. Log in to the Primary Correlation server to check IP address using ifconfig command.
    
3. Gather the correct ip address to reconfigure in NwServer using the below command.

   security-cli-client --set-config-prop --prop-hierarchy nw --prop-name rsa.data.application.servers[0] --prop-value
   
   Note: Primary Correlation server ip must be gathered from Step2. sample command will be as below.
   security-cli-client --set-config-prop --prop-hierarchy nw --prop-name rsa.data.application.servers[0] --prop-value 10.11.5.5

    
4. Re-run Step1 command to see correct Correlation Server ip.
   [root@SA ]# security-cli-client --get-config-prop --prop-hierarchy nw --prop-name rsa.data.application.servers[0]
   2020-02-07 09:31:50.560 INFO 35150 --- [ main] Bootstrap : Service logs will be written to /var/log/netwitness/security-client
   2020-02-07 09:31:50.568 INFO 35150 --- [ main] Bootstrap : Service configuration will be read from /etc/netwitness/security-client
   2020-02-07 09:31:50.661 INFO 35150 --- [ main] Bootstrap : Starting security-client.dbedfafb-ca48-4577-94e3-cd4816c0b93f (v0.0.0.0)
   2020-02-07 09:31:50.869 INFO 35150 --- [ main] Bootstrap : Initialized service cryptography with 4 providers (BSAFE=CRYPTOJ 6.2.4.0.1 20180724 0958, FIPS-140=true).
   2020-02-07 09:31:51.605 INFO 35150 --- [ main] c.r.n.i.s.client.SecurityApplication : Starting SecurityApplication on SA with PID 35150 (/usr/bin/security-cli-client.jar started by root in /var/log/netwitness/respond-server)
   2020-02-07 09:31:51.605 INFO 35150 --- [ main] c.r.n.i.s.client.SecurityApplication : The following profiles are active: amqp
   2020-02-07 09:31:51.687 INFO 35150 --- [ main] Bootstrap : Service will accept AMQP requests at broker localhost:5672/rsa/system
   2020-02-07 09:31:51.688 INFO 35150 --- [ main] Bootstrap : Service will use the deployment security-server
   2020-02-07 09:31:53.054 WARN 35150 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.LocalDateTime to class java.time.Instant as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:31:53.055 WARN 35150 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.Instant to class java.time.LocalDateTime as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:31:53.107 WARN 35150 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.LocalDateTime to class java.time.Instant as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:31:53.107 WARN 35150 --- [ main] o.s.data.convert.CustomConversions : Registering converter from class java.time.Instant to class java.time.LocalDateTime as reading converter although it does not convert from a store-supported type! You might want to check you annotation setup at the converter implementation.
   2020-02-07 09:31:53.306 INFO 35150 --- [ main] c.r.n.i.s.client.SecurityApplication : Started SecurityApplication in 3.503 seconds (JVM running for 3.917)
   2020-02-07 09:31:53.719 INFO 35150 --- [shake Completed] Security : Accepted new connection with CN=20b26165-dd2d-4b9d-8189-4d8b2956347b,OU=NetWitness Platform,O=RSA,L=Reston,ST=VA,C=US from 20b26165-dd2d-4b9d-8189-4d8b2956347b using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
   2020-02-07 09:31:53.871 INFO 35150 --- [ main] c.r.n.i.security.client.SecurityClient : Retrieving value from Config-Server for property: nw:rsa.data.application.servers[0]
   10.11.5.5
   2020-02-07 09:31:53.929 INFO 35150 --- [ main] SystemOperation : Update current versions on disk {com.rsa.asoc.compass.config-api=5.14.0, com.rsa.asoc.compass.orchestration-api=5.12.0, com.rsa.asoc.compass.security-api=5.13.0}
   2020-02-07 09:31:56.188 INFO 35150 --- [ main] c.r.n.i.s.client.SecurityApplication : Tasks completed successfully...
   [2020-02-07T09:31:56+00:00] <35138> (INFO) Request completed successfully.
   
    
5. Run systemctl restart rsa-nw-respond-server.service command and verify GUI ADMIN->Services page to see Respond service online.

Product Details

Summary

This document outlines the procedure to bring Respond service online by correcting Correct Correlation ip.

Approval Reviewer Queue

RSA NetWitness Suite Approval Queue