Skip to content
  • There are no suggestions because the search field is empty.

nwbroker.service is unable to start in RSA NetWitness Platform 11.3.x

Issue

NwBroker service was failing to start as shown below in NW 11.3.x.

# service nwbroker status
Redirecting to /bin/systemctl status nwbroker.service
   nwbroker.service - Netwitness Broker
   Loaded: loaded (/usr/lib/systemd/system/nwbroker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Wed 2019-08-21 15:20:36 KST; 24h ago
 Main PID: 259920 (code=exited, status=1/FAILURE)

Aug 21 15:20:35 mss-broker1 systemd[1]: Unit nwbroker.service entered failed state.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service failed.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service holdoff time over, scheduling restart.
Aug 21 15:20:35 mss-broker1 systemd[1]: start request repeated too quickly for nwbroker.service
Aug 21 15:20:35 mss-broker1 systemd[1]: Failed to start Netwitness Broker.
Aug 21 15:20:35 mss-broker1 systemd[1]: Unit nwbroker.service entered failed state.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service failed.
Warning: nwbroker.service changed on disk. Run 'systemctl daemon-reload' to reload units.

# /usr/sbin/NwBroker
(i) 2019-Aug-22 15:49:45 [Engine]  RSA NetWitness Service Copyright 2001-2019, RSA Security Inc. All Rights Reserved.
(i) 2019-Aug-22 15:49:45 [Engine]  Running broker in console
(d) 2019-Aug-22 15:49:45 [Engine]  [broker](7f1318d5d940): Entering ServiceBase::Initialize()
(d) 2019-Aug-22 15:49:45 [Engine]  [broker](7f1318d5d940): ServiceBase::SetStatus(Stopped, Start Pending)
(a) 2019-Aug-22 15:49:45 [Engine]  RSA NetWitness Service, Broker 11.3.1.0 (Jun 14 2019) 64 bit Starting
(F) 2019-Aug-22 15:49:45 [Engine]  Failed to start engine because of exception: Throw in function X509* nw::{anonymous}::getX509FromPEM(const boost::filesystem::path&)
Dynamic exception type: boost::exception_detail::clone_impl
std::exception::what: error loading trusted certificate file
[nw::ssl_error_tag*] = error:0E06D06C:configuration file routines:NCONF_get_string:no value error:02001002:system library:fopen:No such file or directory error:2006D080:BIO routines:BIO_new_file:no such file
[boost::errinfo_at_line_*] = 45
[boost::errinfo_file_name_*] = /etc/netwitness/ng/broker/trustpeers/c5al34bl.0
[boost::errinfo_api_function_*] = BIO_new_file

# ls -al /etc/netwitness/ng/broker/trustpeers/ <-- You need to note this information for restoration after pem file regeneration
total 0
drwxr-x---. 2 netwitness netwitness 23 Aug  1 07:14 .
drwxr-x---. 6 netwitness netwitness 90 Aug  1 07:13 ..
lrwxrwxrwx. 1 root       root       67 Aug  1 07:14 fdc2f8fd.0 -> /etc/pki/nw/peer/sa-server/d4edb4d8-3362-4568-991b-ef5d627dea0c.pem

Cause

For some reason, the pem certificate file for the sa-server service id was broken or missing in /etc/pki/nw/peer/sa-server.
In this case, nwbroker service is unable to start.

Resolution

You need to re-generate the pem file for the sa-server service id in this case.

Please follow the steps below.
  1. Get ssl certificate information and save it to file('root.out')
    # openssl s_client -connect localhost:7000 -tls1_2 > root.out
     
  2. Edit it using vi, then extract content and save it as a d4edb4d8-3362-4568-991b-ef5d627dea0c.pem file.
    Note: certificate is the copy of the section:

    -----BEGIN CERTIFICATE-----
    to
    -----END CERTIFICATE-----
     

  3. Locate pem file into /etc/pki/nw/peer/sa-server/ and link it same as before.
    # ls -al /etc/netwitness/ng/broker/trustpeers
    total 0
    drwxr-x---. 2 netwitness netwitness 23 Aug  1 07:14 .
    drwxr-x---. 6 netwitness netwitness 90 Aug  1 07:13 ..
    lrwxrwxrwx. 1 root       root       67 Aug  1 07:14 fdc2f8fd.0 -> /etc/pki/nw/peer/sa-server/d4edb4d8-3362-4568-991b-ef5d627dea0c.pem

    Once completed, you are able to start nwbroker.service without any issue.

Internal Comments

SACE-12047


Product Details

RSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.3.1.0
Platform: CentOS 7

Summary

NwBroker service was failing to start due to broken or missing pem certificate file.


Approval Reviewer Queue

RSA NetWitness Suite Approval Queue