NetWitness platform Source-server Service Crashes Following Upgrade to Version 12.4.0.0
Issue
The Source-server service may crash after upgrading to RSA NetWitness Platform version 12.4.0.0The following error logs are observed in the /var/log/netwitness/source-server/source-server.log :
2024-04-29 11:32:23,361 [ main] ERROR Upgrade|
Failed to Update contentName state should be: writes is not an empty list
2024-04-29 11:32:23,768 [ main] INFO Upgrade|Start of separating custom xml parser from existing log devices in content library
2024-04-29 11:32:23,823 [ main] ERROR o.s.b.SpringApplication|Application run failed
java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details:
Bean [com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor]
Method [public void com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.onApplicationEvent(org.springframework.boot.context.event.ApplicationReadyEvent) throws com.rsa.asoc.launch.engine.lifecycle.UpgradeFailedException]
Resolved arguments:
[0] [type=org.springframework.boot.context.event.ApplicationReadyEvent] [value=org.springframework.boot.context.event.ApplicationReadyEvent[source=org.springframework.boot.SpringApplication@7db791df]]
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:233)
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:445)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378)
at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109)
at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:343)
at com.rsa.asoc.launch.engine.Launch.<init>(Launch.java:33)
at com.rsa.asoc.launch.engine.Launch.main(Launch.java:43)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: com.rsa.asoc.launch.engine.lifecycle.UpgradeFailedException: java.lang.NullPointerException
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.performUpgrade(UpgradeComponentVisitor.java:100)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.onApplicationEvent(UpgradeComponentVisitor.java:59)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:348)
... 24 common frames omitted
Caused by: java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:209)
at com.rsa.netwitness.usm.server.central.utils.ContentUtil.extractBaseParserFile(ContentUtil.java:1323)
at com.rsa.netwitness.usm.server.upgrade.central.CentralMgmtUpgradeComponent.extractCustomLogDeviceContent(CentralMgmtUpgradeComponent.java:360)
at com.rsa.netwitness.usm.server.upgrade.central.CentralMgmtUpgradeComponent.performUpgrade(CentralMgmtUpgradeComponent.java:214)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.lambda$performUpgrade$0(UpgradeComponentVisitor.java:94)
at java.base/java.util.Optional.ifPresent(Optional.java:178)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.performUpgrade(UpgradeComponentVisitor.java:87)
... 30 common frames omitted
2024-04-29 11:32:23,768 [ main] INFO Upgrade|Start of separating custom xml parser from existing log devices in content library
2024-04-29 11:32:23,823 [ main] ERROR o.s.b.SpringApplication|Application run failed
java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details:
Bean [com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor]
Method [public void com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.onApplicationEvent(org.springframework.boot.context.event.ApplicationReadyEvent) throws com.rsa.asoc.launch.engine.lifecycle.UpgradeFailedException]
Resolved arguments:
[0] [type=org.springframework.boot.context.event.ApplicationReadyEvent] [value=org.springframework.boot.context.event.ApplicationReadyEvent[source=org.springframework.boot.SpringApplication@7db791df]]
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:233)
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:445)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378)
at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109)
at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:343)
at com.rsa.asoc.launch.engine.Launch.<init>(Launch.java:33)
at com.rsa.asoc.launch.engine.Launch.main(Launch.java:43)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: com.rsa.asoc.launch.engine.lifecycle.UpgradeFailedException: java.lang.NullPointerException
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.performUpgrade(UpgradeComponentVisitor.java:100)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.onApplicationEvent(UpgradeComponentVisitor.java:59)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:348)
... 24 common frames omitted
Caused by: java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:209)
at com.rsa.netwitness.usm.server.central.utils.ContentUtil.extractBaseParserFile(ContentUtil.java:1323)
at com.rsa.netwitness.usm.server.upgrade.central.CentralMgmtUpgradeComponent.extractCustomLogDeviceContent(CentralMgmtUpgradeComponent.java:360)
at com.rsa.netwitness.usm.server.upgrade.central.CentralMgmtUpgradeComponent.performUpgrade(CentralMgmtUpgradeComponent.java:214)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.lambda$performUpgrade$0(UpgradeComponentVisitor.java:94)
at java.base/java.util.Optional.ifPresent(Optional.java:178)
at com.rsa.asoc.launch.engine.lifecycle.UpgradeComponentVisitor.performUpgrade(UpgradeComponentVisitor.java:87)
... 30 common frames omitted
PreUpgrade check tool can indicate below error for this issue:
Cause
If custom Log Devices uploaded using the CCM-UI page in a previous version. This issue occurs because the upgrade process expects specific file formats (.xml and -custom.xml files) for these custom Log Devices, and any deviations can lead to crashes.
Resolution
The users have to recreate the .envision file with the proper structure:
envision File Structure:
/etc/netwitness/ng/envision/etc/devices/<LogDevice Name>/<LogDevice>.xml
For example:
/etc/netwitness/ng/envision/etc/devices/accurev/accurevmsg.xml
Please refer below document for additional details: Import Content to Content Library
Screenshot when upgrade is successful:
Product Details
NetWitness Product Set: NetWitness PlatformNetWitness Product/Service Type: Source Server, Log decoder
NetWitness Version/Condition: 12.4 or later
Platform: CentOS/ Alma Linux
Approval Reviewer Queue
Technical approval queue