Skip to content
  • There are no suggestions because the search field is empty.

RSA NetWitness NwHost update failing as Orchestration database missing installedServices and Version

Issue

When update initiated with the below command that fails with orchestration-server errors as below.

upgrade-cli-client –-upgrade --host-addr --version 11.X.X


On Host /var/log/netwitness/config-management/chef-solo.log:
[2020-09-22T08:49:54+00:00] INFO: Start handlers complete.
[2020-09-22T08:49:54+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found:
[2020-09-22T08:49:55+00:00] INFO: Loading cookbooks [nw-repositories@0.7.1, nw-pre-install@0.3.1, nw-post-install@0.3.2, yum@3.12.0, nw-base@0.3.1, systemd@2.1.3, nw-rabbitmq@0.5.6, nw-mongo@0.9.7, line@1.1.1, nw-pki@0.2.0, salt-master@0.3.3]
[2020-09-22T08:49:56+00:00] WARN: Chef::Provider::YumRepository already exists! Cannot create deprecation class for LWRP provider yum_repository from cookbook yum
[2020-09-22T08:49:56+00:00] WARN: YumRepository already exists! Deprecation class overwrites Custom resource yum_repository from cookbook yum
[2020-09-22T08:49:56+00:00] ERROR: Running exception handlers
[2020-09-22T08:49:56+00:00] ERROR: Exception handlers complete
[2020-09-22T08:49:56+00:00] FATAL: Stacktrace dumped to /var/lib/netwitness/config-management/cache/chef-stacktrace.out
[2020-09-22T08:49:56+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-09-22T08:49:56+00:00] ERROR: undefined method `[]' for nil:NilClass
[2020-09-22T08:49:56+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

On NwServer /var/log/netwitness/orchestration-server/orchestration-server.log:
================================================================================
Recipe Compile Error in /var/lib/netwitness/config-management/cache/cookbooks/nw-rabbitmq/resources/user.rb
================================================================================

NoMethodError
-------------
undefined method `[]' for nil:NilClass

Cookbook Trace:
---------------
/var/lib/netwitness/config-management/cache/cookbooks/nw-rabbitmq/resources/user.rb:45:in `class_from_file'

Relevant File Content:
----------------------
/var/lib/netwitness/config-management/cache/cookbooks/nw-rabbitmq/resources/user.rb:

38:
39: attribute :connection_host, String,
40: default: NwBase::Global.get_rabbitmq_hostname
41: attribute :connection_port, Integer, default: node['nw-rabbitmq']['mgmt_port']
42: attribute :connection_user, String, default: node['nw-rabbitmq']['admin_user']
43: attribute :connection_pass, String, default: 'wrong_password'
44: attribute :connection_vhost, String, default: '/rsa/system'
45>> attribute :connection_ca_file, String, default: node['nw-pki']['ca']['cert_pem']
46:
47: attribute :connection_data
48:
49: action_class do
50: # rubocop:disable Metrics/MethodLength
51: def user_exists?(name)
52: cmd = "rabbitmqctl -q list_users |grep '^#{name}\\s'"
53: cmd = Mixlib::ShellOut.new(cmd, env: { 'HOME' => '/var/lib/rabbitmq' })
54: cmd.run_command

Platform:
---------
x86_64-linux

Cause

This is due to orchestration database missing installedServices and Version information.  Please verify using below commands in NwServer.

#mongo admin -u deploy_admin -p xxxxx

> use orchestration-server

> db.host.find({ "_id" : "77e4d0c9-6e25-4a44-996f-78afbae7e16d" }).pretty()
{
"_id" : "77e4d0c9-6e25-4a44-996f-78afbae7e16d",
"hostname" : "10.10.10.10",
"displayName" : "10.10.10.10",
"thirdParty" : false,
"installedServices" : [ ],
"meta" : {

},
"_class" : "com.rsa.asoc.orchestration.host.HostEntity"
}

Workaround

Please follow the below steps to add Version and InstalledServicesdetails in the mongo database of NwServer for NwHost.
  1. Backup mongo db before performing the steps (Substitute correct password for the -p value).


    # mongoexport --db orchestration-server --collection host --out host.json -u deploy_admin -p xxxxx --authenticationDatabase admin

  2. Run the below commands in sequence.
     Note: Node id of Nwhost noted using cat /etc/salt/minion command in Nwhost.


    #mongo admin -u deploy_admin -p xxxxx
    MongoDB shell version v3.6.3
    connecting to: mongodb://127.0.0.1:27017/admin
    MongoDB server version: 3.6.3

    > use orchestration-server
    switched to db orchestration-server

    >db.host.update({ "_id" : "77e4d0c9-6e25-4a44-996f-78afbae7e16d" }, { $set: { "installedServices" : ["LogHybrid"]}})

    >db.host.update({ "_id" : "77e4d0c9-6e25-4a44-996f-78afbae7e16d" }, { $set: { "version" : { "major" : 11, "minor" : 3, "servicePack" : 2, "patch" : 1, "snapshot" : false, "rawVersion" : "11.3.2.1" }}})
  3. Verify details using >db.host.find({ "_id" : "77e4d0c9-6e25-4a44-996f-78afbae7e16d" }).pretty()
     
Expected output:
{
"_id" : " 77e4d0c9-6e25-4a44-996f-78afbae7e16d",
"_class" : "com.rsa.asoc.orchestration.host.HostEntity",
"hostname" : "10.10.10.10",
"displayName" : "10.10.10.10",
"version" : {
"major" : 11,
"minor" : 3,
"servicePack" : 2,
"patch" : 1,
"snapshot" : false,
"rawVersion" : "11.3.2.1"
},
"thirdParty" : false,
"installedServices" : [
"LogHybrid"
],
"meta" : {

}
}
4. Try the update again and this should start update now.

Product Details

RSA Product Set: RSA NetWitness Logs & Network
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.3.2.1
Platform: CentOS
O/S Version: 7

Summary

This document outlines the procedure add orchestration details in mongo database manually.


Approval Reviewer Queue

RSA NetWitness Suite Approval Queue