Failed to find node zero record Error in RSA NetWitness 11
Issue
Indications of the error :- nwsetup-tui fails to run
- Orchestration fails to run on hosts
- Fails to update NW Admin node
[root@NWSRV] ~# orchestration-cli-client --update-admin-node
2019-08-19 00:54:22.439 INFO 12781 --- [ main] Bootstrap : Service logs will be written to /var/log/netwitness/orchestration-client
2019-08-19 00:54:22.446 INFO 12781 --- [ main] Bootstrap : Service configuration will be read from /etc/netwitness/orchestration-client
2019-08-19 00:54:22.645 INFO 12781 --- [ main] Bootstrap : Starting orchestration-client.ba639be0-6e9a-4623-9178-b1a2274c415e (v0.0.0.0)
2019-08-19 00:54:23.271 INFO 12781 --- [ main] Bootstrap : Initialized service cryptography with 4 providers (BSAFE=CRYPTOJ 6.2.2 20161215 0745, FIPS-140=true).
2019-08-19 00:54:24.142 INFO 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : Starting OrchestrationApplication on AUNWSRV with PID 12781 (/usr/bin/orchestration-cli-client.jar started by root in /root)
2019-08-19 00:54:24.142 INFO 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : The following profiles are active: standard
2019-08-19 00:54:24.245 INFO 12781 --- [ main] Bootstrap : Service will accept AMQP requests at broker localhost:5672/rsa/system
2019-08-19 00:54:24.249 INFO 12781 --- [ main] Bootstrap : Service will use the deployment security-server
2019-08-19 00:54:25.768 INFO 12781 --- [shake Completed] Security : Accepted new connection with CN=c2886414-211a-4e22-86f8-dfcb448e26eb,OU=NetWitness Platform,O=RSA,L=Reston,ST=VA,C=US from localhost using TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
2019-08-19 00:54:26.876 ERROR 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : Exception processing request
java.lang.IllegalStateException: Failed to find node zero record
[2019-08-19T00:54:27+00:00] <12778> (ERROR) Failed, aborting...
2019-08-19 00:54:22.439 INFO 12781 --- [ main] Bootstrap : Service logs will be written to /var/log/netwitness/orchestration-client
2019-08-19 00:54:22.446 INFO 12781 --- [ main] Bootstrap : Service configuration will be read from /etc/netwitness/orchestration-client
2019-08-19 00:54:22.645 INFO 12781 --- [ main] Bootstrap : Starting orchestration-client.ba639be0-6e9a-4623-9178-b1a2274c415e (v0.0.0.0)
2019-08-19 00:54:23.271 INFO 12781 --- [ main] Bootstrap : Initialized service cryptography with 4 providers (BSAFE=CRYPTOJ 6.2.2 20161215 0745, FIPS-140=true).
2019-08-19 00:54:24.142 INFO 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : Starting OrchestrationApplication on AUNWSRV with PID 12781 (/usr/bin/orchestration-cli-client.jar started by root in /root)
2019-08-19 00:54:24.142 INFO 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : The following profiles are active: standard
2019-08-19 00:54:24.245 INFO 12781 --- [ main] Bootstrap : Service will accept AMQP requests at broker localhost:5672/rsa/system
2019-08-19 00:54:24.249 INFO 12781 --- [ main] Bootstrap : Service will use the deployment security-server
2019-08-19 00:54:25.768 INFO 12781 --- [shake Completed] Security : Accepted new connection with CN=c2886414-211a-4e22-86f8-dfcb448e26eb,OU=NetWitness Platform,O=RSA,L=Reston,ST=VA,C=US from localhost using TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
2019-08-19 00:54:26.876 ERROR 12781 --- [ main] c.r.n.i.o.c.OrchestrationApplication : Exception processing request
java.lang.IllegalStateException: Failed to find node zero record
[2019-08-19T00:54:27+00:00] <12778> (ERROR) Failed, aborting...
Cause
For some reason, node-zero meta was not updated in the mongo db on the first time when NW Server was installed.
Resolution
Update the meta record manually in the mongo db.- Backup mongo db before performing the steps (Substitute correct password for the -p value).
# mongoexport --db orchestration-server --collection host --out host.json -u deploy_admin -p xxxxx --authenticationDatabase admin
- Log in mongo admin.
[root@NWSRV] ~# mongo admin -u deploy_admin -p xxxxx
MongoDB shell version v3.6.4
connecting to: mongodb://127.0.0.1:27017/admin
MongoDB server version: 3.6.4
> use orchestration-server
switched to db orchestration-server
> show collections
groups
host
job
service
task - Confirm if meta record is empty.
> db.host.find()
{ "_id" : "c2886414-211a-4e22-86f8-dfcb448e26eb", "_class" : "com.rsa.asoc.orchestration.host.HostEntity", "hostname" : "x.x.x.x", "displayName" : "NWSRV", "version" : { "major" : 11, "minor" : 2, "servicePack" : 1, "patch" : 1, "snapshot" : false, "rawVersion" : "11.2.1.1" }, "thirdParty" : false, "installedServices" : [ "AdminServer" ], "meta" : { } } - Update the meta record manually with "node-zero:true"
> db.host.update({ "_id" : "c2886414-211a-4e22-86f8-dfcb448e26eb" }, { $set: { "meta" : { "node-zero" : true }}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.host.find()
{ "_id" : "c2886414-211a-4e22-86f8-dfcb448e26eb", "_class" : "com.rsa.asoc.orchestration.host.HostEntity", "hostname" : "x.x.x.x", "displayName" : "NWSRV", "version" : { "major" : 11, "minor" : 2, "servicePack" : 1, "patch" : 1, "snapshot" : false, "rawVersion" : "11.2.1.1" }, "thirdParty" : false, "installedServices" : [ "AdminServer" ], "meta" : { "node-zero" : true } }
Product Details
RSA Product Set: NetWitness Logs & NetworkRSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.x
Platform: CentOS 7
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue