RSA NetWitness Endpoint Files display has a negative On Hosts count
Issue
The NetWitness UI, Investigate > Files shows some negative value in the "On Hosts" field.
Cause
It does not make sense to show a file on a negative number of hosts.There are three possibilities when the Files "On Host" count can decrement.
- When the Endpoint agent migrated across multiple Endpoint servers.
- Files getting deleted on an Endpointn Agent machine.
- Data deletion as part of the data deletion cleanup job.
At this time which of these actions is causing the negative number is still under investigation.
A change will be added into the future NetWitness 11.6 version to stop negative "On Host" values.
Workaround
-
Wait for the Files entries with a negative "On Host" count to be deleted.
By default in the NetWitness UI, Admin > Services > Endpoint Server, Config, Data Retention Scheduler is configured to delete data older than 30 days.
The Files entries with a negative "On Host" count will be deleted after data retention days is exceeded.
Reference: Endpoint Config: Data Retention Scheduler TabOr
-
Manually delete the mongodb entries for files with a negative "On Host" count.
Note: This does not stop a negative "On Host" count for files appearing again in the future.
- ssh login to the Endpoint Server appliance
Create a backup of the endpoint-server mongodb collection "file".
mongodump -u deploy_admin -p netwitness --authenticationDatabase=admin --port 27017 --db=endpoint-server --collection file --gzip --out=/root/mongodumpgz
Substitute netwitness with the correct deploy_admin password.
For example:
mongodump -u deploy_admin -p netwitness --authenticationDatabase=admin --port 27017 --db=endpoint-server --collection file --gzip --out=/root/mongodumpgz
Substitute netwitness with the correct deploy_admin password.
For example:
[root@Endpoint ~]#
mongodump -u deploy_admin -p netwitness --authenticationDatabase=admin --port 27017 --db=endpoint-server --collection file --gzip --out=/root/mongodumpgz
2020-11-13T03:58:21.712+0000 writing endpoint-server.file to
2020-11-13T03:58:24.475+0000 [#####...................] endpoint-server.file 9811/44533 (22.0%)
2020-11-13T03:58:27.616+0000 [######..................] endpoint-server.file 11147/44533 (25.0%)
2020-11-13T03:58:30.426+0000 [##############..........] endpoint-server.file 26282/44533 (59.0%)
2020-11-13T03:58:33.439+0000 [###############.........] endpoint-server.file 28403/44533 (63.8%)
2020-11-13T03:58:36.441+0000 [################........] endpoint-server.file 30506/44533 (68.5%)
2020-11-13T03:58:39.472+0000 [#################.......] endpoint-server.file 31838/44533 (71.5%)
2020-11-13T03:58:42.428+0000 [##################......] endpoint-server.file 33461/44533 (75.1%)
2020-11-13T03:58:45.550+0000 [####################....] endpoint-server.file 38941/44533 (87.4%)
2020-11-13T03:58:48.427+0000 [#######################.] endpoint-server.file 43000/44533 (96.6%)
2020-11-13T03:58:49.519+0000 [########################] endpoint-server.file 44533/44533 (100.0%)
2020-11-13T03:58:49.520+0000 done dumping endpoint-server.file (44533 documents)
2020-11-13T03:58:21.712+0000 writing endpoint-server.file to
2020-11-13T03:58:24.475+0000 [#####...................] endpoint-server.file 9811/44533 (22.0%)
2020-11-13T03:58:27.616+0000 [######..................] endpoint-server.file 11147/44533 (25.0%)
2020-11-13T03:58:30.426+0000 [##############..........] endpoint-server.file 26282/44533 (59.0%)
2020-11-13T03:58:33.439+0000 [###############.........] endpoint-server.file 28403/44533 (63.8%)
2020-11-13T03:58:36.441+0000 [################........] endpoint-server.file 30506/44533 (68.5%)
2020-11-13T03:58:39.472+0000 [#################.......] endpoint-server.file 31838/44533 (71.5%)
2020-11-13T03:58:42.428+0000 [##################......] endpoint-server.file 33461/44533 (75.1%)
2020-11-13T03:58:45.550+0000 [####################....] endpoint-server.file 38941/44533 (87.4%)
2020-11-13T03:58:48.427+0000 [#######################.] endpoint-server.file 43000/44533 (96.6%)
2020-11-13T03:58:49.519+0000 [########################] endpoint-server.file 44533/44533 (100.0%)
2020-11-13T03:58:49.520+0000 done dumping endpoint-server.file (44533 documents)
- Use the below mongodb commands to remove entries with a negative count on the Endpoint Server.
mongo -u deploy_admin -p netwitness endpoint-server --authenticationDatabase=admin
db.file.find({hostCount : {$lt : 0}}).count() /* Count how many files have negative hostCount */;
db.file.find({hostCount : {$lt : 0}}) /* Show the file entries with negative hostCount */;
db.file.remove({hostCount : {$lt : 0}}) /* Remove file entries with negative hostCount */;
exit
Substitute netwitness with the correct deploy_admin password
For example:
db.file.find({hostCount : {$lt : 0}}).count() /* Count how many files have negative hostCount */;
db.file.find({hostCount : {$lt : 0}}) /* Show the file entries with negative hostCount */;
db.file.remove({hostCount : {$lt : 0}}) /* Remove file entries with negative hostCount */;
exit
Substitute netwitness with the correct deploy_admin password
For example:
[root@Endpoint ~]#
mongo -u deploy_admin -p netwitness endpoint-server --authenticationDatabase=admin
MongoDB shell version v4.0.13
connecting to: mongodb://127.0.0.1:27017/endpoint-server?authSource=admin&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8063ee15-abca-4406-81d1-7ddd930fac41") }
MongoDB server version: 4.0.13
> db.file.find({hostCount : {$lt : 0}}).count() /* Count how many files have negative hostCount */;
1
> db.file.find({hostCount : {$lt : 0}}) /* Show the file entries with negative hostCount */;
{ "_id" : "48d4fde31b9d7c3024e77387549feb8857e2632babee9e6a4bb03a61ccd4bdd0", "machineOsType" : "windows", "format" : "script", "checksumSha256" : "48d4fde31b9d7c3024e77387549feb8857e2632babee9e6a4bb03a61ccd4bdd0", "checksumSha1" : "c12950ffda47860d94b33eaa7ba8b2453f5d7332", "checksumMd5" : "000016fc005cde39e2c27eddee096467", "entropy" : 0, "size" : 128, "firstFileName" : "[FILELESS_SCRIPT_000016FC005CDE39E2C27EDDEE096467]", "script" : { }, "firstSeenTime" : ISODate("2020-07-27T08:20:37.740Z"), "fileStatus" : "Neutral", "hostCount" : -1, "score" : 0, "reputationStatus" : "Unknown" }
> db.file.remove({hostCount : {$lt : 0}}) /* Remove file entries with negative hostCount */;
WriteResult({ "nRemoved" : 1 })
> exit
bye
MongoDB shell version v4.0.13
connecting to: mongodb://127.0.0.1:27017/endpoint-server?authSource=admin&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8063ee15-abca-4406-81d1-7ddd930fac41") }
MongoDB server version: 4.0.13
> db.file.find({hostCount : {$lt : 0}}).count() /* Count how many files have negative hostCount */;
1
> db.file.find({hostCount : {$lt : 0}}) /* Show the file entries with negative hostCount */;
{ "_id" : "48d4fde31b9d7c3024e77387549feb8857e2632babee9e6a4bb03a61ccd4bdd0", "machineOsType" : "windows", "format" : "script", "checksumSha256" : "48d4fde31b9d7c3024e77387549feb8857e2632babee9e6a4bb03a61ccd4bdd0", "checksumSha1" : "c12950ffda47860d94b33eaa7ba8b2453f5d7332", "checksumMd5" : "000016fc005cde39e2c27eddee096467", "entropy" : 0, "size" : 128, "firstFileName" : "[FILELESS_SCRIPT_000016FC005CDE39E2C27EDDEE096467]", "script" : { }, "firstSeenTime" : ISODate("2020-07-27T08:20:37.740Z"), "fileStatus" : "Neutral", "hostCount" : -1, "score" : 0, "reputationStatus" : "Unknown" }
> db.file.remove({hostCount : {$lt : 0}}) /* Remove file entries with negative hostCount */;
WriteResult({ "nRemoved" : 1 })
> exit
bye
- Refresh the NetWitness UI webpage.
Refresh the NetWitness UI, Investigate > Files, and the entries with a negative value in the "On Hosts" field are now removed.
Product Details
RSA Product Set: RSA NetWitness PlatformRSA Product/Service Type: Endpoint Server
RSA Version/Condition: 11.3, 11.4, 11.5
Platform: CentOS
O/S Version: 7
Summary
The NetWitness UI, Investigate > Files shows with some negative On Hosts count.
Approval Reviewer Queue
RSA NetWitness Suite Approval Queue