.png?width=180&height=32&name=NW%20Logo%20(2).png){kind=small}
Implementing the RSA NetWitness Platform with Dell EMC Unity Storage
- Who should read this document
- Prerequisites
- Overview
- Install the Appliance
- Allocate Storage in Unity
- RAID Groups needed for the RSA NetWitness Platform
- Performance Recommendations
- Recommended Storage Allocation Per Appliance
- Install Dell EMC PowerPath on the Appliance
- Set up the Appliance as an RSA NetWitness Node
- Run the RSA NetWitness Unity Configuration Program
- Invoking NwArrayConfig
- Troubleshooting
- Common Error Output from NwArrayConfig
- Unity-specific Issues You May Encounter During Installation
- Replace Unity Storage with JBOD (That is, DAC or PowerVault)
Who should read this document
System administrators installing RSA NetWitness appliances that are attached to Dell EMC Unity storage.
Prerequisites
- Dell EMC Unity storage is and accessible
- Dell EMC PowerPath licenses are available.
Note: Use the latest version of PowerPath. - An RSA NetWitness appliance is available that can be attached to Dell EMC Unity storage:
- Packet Decoder
- Log Decoder
- Concentrator
- Archiver
Overview
It is important to follow the steps in the order listed below.
- Install the appliance software using the OS installation image
- Allocate storage through the Dell EMC Unity UI
- Install PowerPath on the appliance
- Instantiate the appliance with the RSA NetWitness Platform software
- Run the RSA NetWitness Unity configuration program
Install the Appliance
If your appliance does not already have the RSA NetWitness Platform 11.1 or 10.6.5 base OS image on it yet, you may have to install it using the RSA-provided OS image. Further details on installing the RSA NetWitness Platform install image are located in the following documents:
- RSA NetWitness Logs & NetWitness 10.6.5.x to 11.1 Physical Host Upgrade Guide
- RSA Security Analytics 10.6.5 Update Guide
For the purpose of a Unity storage install, we will perform storage setup BEFORE setting up the appliance as an RSA NetWitness node.
Set up the Unity storage before running nwsetup-tui
Allocate Storage in Unity
You will need to work with your Dell EMC Storage Engineer to allocate storage within your Unity environment for the RSA NetWitness Platform.
RAID Groups needed for the RSA NetWitness Platform
Your Unity Storage will contain some number of NL-SAS drives and SSD drives, which may vary depending on the exact configuration purchased. For the purposes of the RSA NetWitness Platform, we recommend organizing RAID groups that correspond to each drive type.
- RAID Group Type: NL-SAS
- Suitable For:
All Packet Decoder volumes
All Log Decoder volumes
All Archiver volumesConcentrator meta volume
- RAID Group Type: SSD
- Suitable For: Concentrator index volume
Performance Recommendations
RSA recommends that Packet and Log Decoders receive two LUNs, one for Packet data, the other for all other databases. This allows you to segregate the high-bandwidth Packet Database from the other databases so they do not compete for I/O bandwidth with other activity.
Concentrators require a separate SSD-based index volume for best performance. This will necessarily be housed on a different RAID group than the Concentrator Meta database volume, which can be stored on NL-SAS. Archivers can utilize a single large NL-SAS storage volume per appliance.
Recommended Storage Allocation Per Appliance
- Appliance Type: Decoder
- First LUN: Meta/Session Volume (smaller NL-SAS volume)
- Second LUN: Packet Volume (large NL-SAS)
- Appliance Type: Log Decoder
- First LUN: Meta/Session Volume (medium-sized NL-SAS)
- Second LUN: Packet Volume (medium-sized NL-SAS)
- Appliance Type: Concentrator
- First LUN: Meta Volume (large NL-SAS)
- Second LUN: Index Volume (SSD)
- Appliance Type: Archiver
- First LUN: Data Archive Volume (large-NL-SAS)
- Second LUN: Not used
Every RSA NetWitness appliance that will be using the Unity storage needs to be added as a host within the Unity interface. After hosts and LUNs are created, you must assign the LUNs to the hosts. Assigning the LUNs to the hosts makes the storage visible to the host. At this point the host will be able to locate the storage through the host-based Dell EMC PowerPath software.
Install Dell EMC PowerPath on the Appliance
Dell EMC PowerPath must be installed on the appliance. Work with your Dell EMC Storage Engineer to receive your licenses and install the software.
Verify that the PowerPath license is installed using the emcpreg command:
[root@NWAPPLIANCE24932 ~]# emcpreg -list
Key BQPO-DB4M-VFC2-Q24R-ML9Z-EQTU
Product: PowerPath
Capabilities: AllA Reboot is recommended after installing PowerPath
After the PowerPath install is complete, you may verify that the LUNs are successfully attached to the appliance using the command powermt display dev=all. An example powermt output is shown here:
[root@NWAPPLIANCE24932 ~]# powermt display dev=all
Pseudo name=emcpowera
Unity ID=APM00174407815 [Host_21]
Logical device ID=600601609D9046006996745A46B60AB6 [DecoderSmall01]
state=alive; policy=CLAROpt; queued-IOs=0
Owner: default=SP A, current=SP A Array failover mode: 4
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
13 lpfc sde SP A6 active alive 0 0
12 lpfc sdc SP B6 active alive 0 0
Pseudo name=emcpowerb
Unity ID=APM00174407815 [Host_21]
Logical device ID=600601609D904600BD96745A8040063A [DecoderLarge01]
state=alive; policy=CLAROpt; queued-IOs=0
Owner: default=SP B, current=SP B Array failover mode: 4
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
13 lpfc sdf SP A6 active alive 0 0
12 lpfc sdd SP B6 active alive 0 0Set up the Appliance as an RSA NetWitness Node
Proceed with installation of the RSA NetWitness Platform software by running nwsetup-tui. Further details can be found in the RSA NetWitness Logs & Packets 11.0 Physical Host Installation Guide
Run the RSA NetWitness Unity Configuration Program
Run NwArrayConfig.py to allocate the storage volumes presented to your appliance within the RSA NetWitness Platform software.
This utility performs these tasks automatically:
- Identifies PowerPath-controlled LUNs presented to the appliance.
- Builds Linux volumes and filesystems
- Creates mount points for the volumes
- Updates the RSA NetWitness service configurations to utilize the provided volumes
Invoking NwArrayConfig
/opt/rsa/saTools/NwArrayConfig.pyBelow is an example of successful output from NwArrayConfig successfully configuring Log Decoder storage:
[root@NWAPPLIANCE24932 ~]# /opt/rsa/saTools/NwArrayConfig.py
Creating new volume group logdecodersmall on /dev/emcpowera
Volume group "logdecodersmall" successfully created
Creating new volume group logdecoder on /dev/emcpowerb
Volume group "logdecoder" successfully created
Success!: Added all available storage found. Successfully configured the logdecoder with the appropriate disk arrays. You will need to restart the logdecoder service for the database configuration to be loaded.Validate that the filesystems have been created and mounted:
[root@NWAPPLIANCE24932 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-root 20G 2.7G 16G 15% /
tmpfs 63G 0 63G 0% /dev/shm
/dev/sda1 496M 67M 403M 15% /boot
/dev/mapper/VolGroup00-usrhome 3.9G 8.1M 3.7G 1% /home
/dev/mapper/VolGroup00-opt 20G 395M 20G 2% /opt
/dev/mapper/VolGroup00-rsaroot 10G 62M 10G 1% /opt/rsa
/dev/mapper/VolGroup00-tmp 20G 33M 20G 1% /tmp
/dev/mapper/VolGroup00-var 20G 112M 20G 1% /var
/dev/mapper/VolGroup01-rabmq 324G 37M 324G 1% /var/lib/rabbitmq
/dev/mapper/VolGroup00-varlog 16G 51M 16G 1% /var/log
/dev/mapper/VolGroup00-nwhome 30G 39M 30G 1% /var/netwitness
/dev/mapper/VolGroup01-lcol 300G 65M 300G 1% /var/netwitness/logcollector
/dev/mapper/VolGroup01-warec 400G 33M 400G 1% /var/netwitness/warehouseconnector
/dev/mapper/VolGroup00-vartmp 4.0G 33M 4.0G 1% /var/tmp
/dev/mapper/logdecodersmall-decoroot 10G 33M 10G 1% /var/netwitness/logdecoder
/dev/mapper/logdecodersmall-index 30G 33M 30G 1% /var/netwitness/logdecoder/index
/dev/mapper/logdecodersmall-metadb 9.0T 34M 9.0T 1% /var/netwitness/logdecoder/metadb
/dev/mapper/logdecodersmall-sessiondb 1.0T 34M 1.0T 1% /var/netwitness/logdecoder/sessiondb
/dev/mapper/logdecoder-packetdb 20T 34M 20T 1% /var/netwitness/logdecoder/packetdbWithin the Core service itself, you can see the storage configuration entries added:
[root@NWAPPLIANCE24932 ~]# NwConsole
RSA Security Analytics Console 10.6.5.0
Copyright 2001-2017, RSA Security Inc. All Rights Reserved.
Type "help" for a list of commands or "man" for a list of manual pages.
> login localhost:50002 admin
Password: **********
Successfully logged in as session 819
[localhost:50002] /> cd /database/config
[localhost:50002] /database/config
[localhost:50002] /database/config> ls
hash.algorithm (Hash Algorithm) = none
hash.databases (Hash Databases) = session,meta,packet
hash.dir (Hash Directory)
manifest.dir (Manifest Directory)
meta.compression (Meta Compression) = none
meta.compression.level (Meta Compression Level) = 0
meta.dir (Meta Database Directory) = /var/netwitness/logdecoder/metadb=8.51 TB
meta.dir.cold (Cold Meta Database Directory)
meta.dir.warm (Warm Meta Database Directory)
meta.file.size (Meta File Size) = 3 GB
meta.files (Meta Open Files) = auto
meta.free.space.min (Meta Minimum Free Space) = 79 GB
meta.index.fidelity (Meta Index Fidelity) = 1
meta.integrity.flush (Meta Integrity Flush) = sync
meta.write.block.size (Meta Write Block Size) = 64 KB
packet.compression (Packet Compression) = none
packet.compression.level (Packet Compression Level) = 0
packet.dir (Packet/Log Database Directory) = /var/netwitness/logdecoder/packetdb=18.99 TB
packet.dir.cold (Cold Packet/Log Database Directory)
packet.dir.warm (Warm Packet/Log Database Directory)
packet.file.size (Packet File Size) = 5 GB
packet.files (Packet Open Files) = auto
packet.free.space.min (Packet Minimum Free Space) = 178 GB
packet.index.fidelity (Packet Index Fidelity) = 1
packet.integrity.flush (Packet Integrity Flush) = sync
packet.write.block.size (Packet Write Block Size) = 64 KB
session.dir (Session Database Directory) = /var/netwitness/logdecoder/sessiondb=972.32 GB
session.dir.cold (Cold Session Database Directory)
session.dir.warm (Warm Session Database Directory)
session.file.size (Session File Size) = 256 MB
session.files (Session Open Files) = auto
session.free.space.min (Session Minimum Free Space) = 8 GB
session.integrity.flush (Session Integrity Flush) = sync
session.write.block.size (Session Write Block Size) = 32 KB
[localhost:50002] /database/config> cd /index/config
[localhost:50002] /index/config
[localhost:50002] /index/config> ls
index.dir (Index Directory) = /var/netwitness/logdecoder/index=26.98 GB
index.dir.cold (Index Cold Storage Directory)
index.dir.warm (Index Warm Storage Directory)
index.slices.open (Index Open Slice Count) = 42
page.compression (Page Compression) = huffhybrid
save.session.count (Save Session Count) = 0� ,,,,,,, ,,,,,,, arrayCfg.log, if it encounters an error in the Unity storage setup. Verbose command output can be found in this file. The arrayCfg.log file is created in the working directory from which the NwArrayConfig.py command is invoked.Common Error Output from NwArrayConfig
- Error: Failed!: Ssl may be set opposite of what was attempted
- Explanation:
The core service configuration could not be updated.
Verify that the core service is running (NwDecoder, NwLogDecoder, NwConcentrator or NwArchiver)
- Error: Failed!: No available VNX LUNs found. Verify VNX configuration before trying again.
- Explanation:
The storage LUNs attached to this system have already been allocated.
As a safety precaution, NwArrayConfig will not overwrite any volume that might currently store data.
To reallocate LUNs, you must unmount any filesystems on them and manually remove the Logical Volumes, Volume Groups, and Physical Volumes defined on the LUNs.
Unity-specific Issues You May Encounter During Installation
- Multiple drives show up in the ‘Failed State’ resulting in the Pool going offline. For example