Certificate Reissue Warning appears after upgrading to RSA NetWitness 11.4 even though the certificates are not to expire for many months
Issue
After upgrading the Admin Server to 11.4, "Certificate Reissue Warning" message box appears as shown below whenever a user logs in to the UI.
Running the ca-expire-test.sh script from Reissue root CA security certificates on RSA NetWitness Platform 11.x confirms that the certificates are to expire in several hundred days.
The issue continues even after reissuing the certificates again by following the same article.
Cause
The issue can occur when "rsa-nw-orchestration-server-cert.pem" file entry is missing from certificate collection on security-server Mongo database.To confirm if this is the case, run an ajax call by browsing to https://
The following output is expected in a working environment.
{"success":true,"data":true}
But the problematic server will show below.
{"success":true,"data":false}
Workaround
--- ADDITIONAL STEPS: follow these steps if the issue persists after performing the above ---
There can be cases where the security-cli-client command returns an error or the issue continues even after following the above steps.
In that case, please ensure db.certificate.find({"subject" : "CN=rsa-nw-orchestration-server"}) returns two entries similar to below output.
> use security-server
> db.certificate.find({"subject" : "CN=rsa-nw-orchestration-server"})
{ "_id" : "rsa-nw-orchestration-server", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "d2e0954a64b79730ba6c2e802fad307797f52d91519eb8f48faa6cffb578d08a", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : "-----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\n
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" }
{ "_id" : "eb74d077-c1f8-48d2-a25a-da34f190db81", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "d2e0954a64b79730ba6c2e802fad307797f52d91519eb8f48faa6cffb578d08a", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : "-----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\n
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" }
If any fields appear to be missing, please follow the steps below to remove and re-add the entries.
> db.certificate.find({"subject" : "CN=rsa-nw-orchestration-server"})
{ "_id" : "rsa-nw-orchestration-server", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "d2e0954a64b79730ba6c2e802fad307797f52d91519eb8f48faa6cffb578d08a", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : "-----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\n
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" }
{ "_id" : "eb74d077-c1f8-48d2-a25a-da34f190db81", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "d2e0954a64b79730ba6c2e802fad307797f52d91519eb8f48faa6cffb578d08a", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : "-----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\n
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" }
- Connect to Mongo DB.
mongo admin -u deploy_admin
use security-server - Delete the entries if exist.
db.certificate.remove({"_id" : "<SERVICE-ID>"}) -- can be skipped if the entry does not existNote: Replace
db.certificate.remove({"_id" : "rsa-nw-orchestration-server"}) -- can be skipped if the entry does not exist.with the actual service ID noted in Step 2 under Resolution.
- Add the entries with all required fields.
db.certificate.insert({ "_id" : "<SERVICE-ID>", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "<FINGERPRINT>", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : " -----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\nNote 1: Replace "-----BEGIN CERTIFICATE----- ...-----END CERTIFICATE-----" with the host's certificate noted in Step 3 under Resolution. Add '\n' to the first and the last lines and '\r\n' to the other lines in the 'cat' command output as shown below example.
(certificate content omitted)MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" })
db.certificate.insert({ "_id" : "rsa-nw-orchestration-server", "subject" : "CN=rsa-nw-orchestration-server", "thumbprint" : "<FINGERPRINT>", "issuer" : "C=US,ST=VA,L=Reston,O=RSA,OU=NetWitness,CN=NetWitness Intermediate CA", "pem" : " -----BEGIN CERTIFICATE-----\nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG\r\n
(certificate content omitted)MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY\r\nNtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M\r\nbNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=\n-----END CERTIFICATE-----", "_class" : "com.rsa.asoc.launch.api.security.pki.Certificate" })
Fromcat /etc/pki/nw/service/rsa-nw-orchestration-server-cert.pemTo
-----BEGIN CERTIFICATE-----
MIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY
NtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M
bNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw=
-----END CERTIFICATE----------BEGIN CERTIFICATE----- \nMIIEaTCCAlGgAwIBAgIUbkocCbaLzqjLOd0KWbDNndWo7bUwDQYJKoZIhvcNAQELBQAwczEjMCEG \r\nNote 2: Replace
(certificate content omitted)
MjFp4FT6ejzpaTioGb7OphJfJBFe6KRO2Y9IkIDfvoh2psvOWBr3NjY8YGAmrSji9kUaPbxOOLEY \r\n
NtMXaZ9Ybu56aXfHE2/PNlfnKQEEbBfQl2qZqTuCWZ9t3ke9RIrbiJbx5MTBPO/+RDlo2HplAm9M \r\n
bNz3VZPJ3uRfECSDYlSHjvJrtH1B/ZRrRrjtOofd8ESLrawSZ+mrGjf6pHTzNYwl/Vw= \n-----END CERTIFICATE-----with the value next to fingerprint= from running the following command. openssl x509 -noout -fingerprint -sha256 -inform pem -in /etc/pki/nw/service/rsa-nw-orchestration-server-cert.pem | sed 's/://g; s/./\L&/g'
- Confirm that the entries are added correctly.
db.certificate.find({"subject" : "CN=rsa-nw-orchestration-server"})
- Log out and log back in to the UI and confirm if the warning is disappeared.
Resolution
To resolve the issue, follow the steps below to add the missing entries to the Mongo database.
- SSH into the Admin Server and back-up certificate collection.
mongoexport -u deploy_admin --authenticationDatabase admin --db security-server --collection certificate --out /root/certificate.json
- Obtain the service ID of the orchestration-server by using one of the following two methods.
Method1:cat /etc/netwitness/orchestration-server/service-idMethod2:echo -e 'use orchestration-server\ndb.service.find({"name" : "orchestration-server"})' | mongo admin -u deploy_admin -p <PASSWORD>
Replacewith the deploy_admin password.
Note the service ID next to "_id" :
- Confirm the .pem file exists with the expected content.
cat /etc/pki/nw/service/rsa-nw-orchestration-server-cert.pemNote that the long string enclosed by -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----.
- Set the registry certificate file.
security-cli-client --set-registry-cert --service-id <SERVICE-ID> --cert-path /etc/pki/nw/service/rsa-nw-orchestration-server-cert.pem -u deploy_admin -k <PASSWORD> -b 127.0.0.1Replace
with the ID noted from step 2 and with the deploy_admin password.
- Check the current setting.
mongo admin -u deploy_adminNote: If only one entry which starts with "{ "_id" : "
(enter password)
use security-server
db.certificate.find({"subject" : "CN=rsa-nw-orchestration-server"})...." is returned, please repeat Step 4 with the following command. security-cli-client --set-registry-cert --service-id rsa-nw-orchestration-server --cert-path /etc/pki/nw/service/rsa-nw-orchestration-server-cert.pem -u deploy_admin -k <PASSWORD> -b 127.0.0.1If both entries(one with UUID and another with rsa-nw-orchestration-server) are returned, please proceed to the next step.
- Log out and log back into the UI and confirm if the warning is disappeared.
Product Details
RSA Product Set: RSA NetWitness PlatformRSA Product/Service Type: Admin Server, UI
RSA Version/Condition: 11.4
Platform: CentOS 7
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue