cert-reissue command fails to run on all hosts in RSA NetWitness 11.x.
Issue
Running cert-reissue per Sys Maintenance: Reissue Certificates fails to run on all NetWitness hosts including the Admin server.The following errors are noticed on the Admin server host.
/var/log/netwitness/config-management/chef-solo.log
Generated at 2020-06-09 17:35:39 +0000
Mixlib::ShellOut::ShellCommandFailed: nw_pki_bootstrap_launch[reissue certs for rsa-nw-license-server] (rsa-license-server::certreissue line 13) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[license-server-get-operational-csr] (/var/lib/netwitness/config-management/cache/cookbooks/nw-pki/resources/bootstrap_launch.rb line 242) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/provider/execute.rb:66:in `rescue in block in action_run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/provider/execute.rb:57:in `block in action_run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/mixin/why_run.rb:51:in `add_action'
/var/log/netwitness/license-server/license-server.log
2020-06-09 18:18:34,011 [unchMessageListenerContainer-2] WARN API|Failure /rsa/license/get-by-service-id [counter=24 reason=AccessDeniedException::Access is denied]
2020-06-09 18:18:38,022 [unchMessageListenerContainer-6] DEBUG FNE_CLIENT|Fetching Host Id http://localhost:3333/api/1.0/hostids/selected
2020-06-09 18:18:38,028 [unchMessageListenerContainer-6] INFO FNE_CLIENT|Found Host Id HostIdentifier(value=C81F66FB27A8, type=ETHERNET)
2020-06-09 18:22:24,049 [unchMessageListenerContainer-4] WARN API|Failure /rsa/security/pki/get-certificate-request [counter=1 reason=AccessDeniedException::Access is denied]
Generated at 2020-06-09 17:35:39 +0000
Mixlib::ShellOut::ShellCommandFailed: nw_pki_bootstrap_launch[reissue certs for rsa-nw-license-server] (rsa-license-server::certreissue line 13) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[license-server-get-operational-csr] (/var/lib/netwitness/config-management/cache/cookbooks/nw-pki/resources/bootstrap_launch.rb line 242) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/provider/execute.rb:66:in `rescue in block in action_run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/provider/execute.rb:57:in `block in action_run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.13.11/lib/chef/mixin/why_run.rb:51:in `add_action'
/var/log/netwitness/license-server/license-server.log
2020-06-09 18:18:34,011 [unchMessageListenerContainer-2] WARN API|Failure /rsa/license/get-by-service-id [counter=24 reason=AccessDeniedException::Access is denied]
2020-06-09 18:18:38,022 [unchMessageListenerContainer-6] DEBUG FNE_CLIENT|Fetching Host Id http://localhost:3333/api/1.0/hostids/selected
2020-06-09 18:18:38,028 [unchMessageListenerContainer-6] INFO FNE_CLIENT|Found Host Id HostIdentifier(value=C81F66FB27A8, type=ETHERNET)
2020-06-09 18:22:24,049 [unchMessageListenerContainer-4] WARN API|Failure /rsa/security/pki/get-certificate-request [counter=1 reason=AccessDeniedException::Access is denied]
Cause
The issue can occur when the Administrators role does not have the correct permission.To confirm if this is the case, please perform the following.
- SSH to the Admin server host.
- Connect to the Mongo database and run the following commands.
mongo admin -u deploy_admin
use security-server
db.getCollection('role').find({"_id" : "Administrators"}) - If the permissions for Administrator role is not set to "*", then this can cause the cert-reissue to fail.
Correct configuration -{ "_id" : "Administrators", "permissions" : [ "*" ], ...Incorrect configuration -{ "_id" : "Administrators", "_class" : "com.rsa.asoc.security.rbac.Role", "description" : "The System Administrators persona is granted all permissions.", "permissions" : [ "respond-server.alertrule.read", "exportList", "content-server.*", "viewEventSources", "contexthub-server.security.read", "content-server.logs.manage", "integration-server.health.read", "esa-analytics-server.logs.manage", ...
Resolution
Perform the followings to resolve the issue.- Back up the current configuration.
mongoexport --db security-server --collection role --out /root/role.json -u deploy_admin -p <PASSWORD>--authenticationDatabase admin
- Update the permission.
mongo admin -u deploy_admin
use security-server
db.getCollection('role').find({"_id" : "Administrators"})
db.getCollection('role').update({"_id" : "Administrators"},{$set:{"permissions": ["*"]}})
db.getCollection('role').find({"_id" : "Administrators"}) -- Confirm the changes
- Run the cert-reissue command.
Product Details
RSA Product Set: RSA NetWitness PlatformRSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.3.x, 11.4.x, 11.5.x.
Platform: CentOS
O/S Version: 7
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue