Cluster Security with NVisionCC: The Forseti Distributed File Integrity Checker

Attackers who are able to compromise a single node in a high performance computing cluster can use that node as a launch point for a number of malicious actions. In many cases, the password used to log into a single node can be used to access a large number of nodes in the system, allowing the attacker to utilize the vast computing and storage capabilities of the compromised cluster to sniff network traffic, carry out brute-force password cracking, launch distributed denial of service attacks, or serve illegal digital content. Often, these types of attackers modify important system files to collect passwords to other accounts, disable certain logging facilities, or create back-doors into the system. In this paper, we present Forseti, a distributed file integrity checker designed specifically for the high performance computing cluster environment. Forseti was designed to address the shortcomings exhibited by existing host-based intrusion detection systems when used in the cluster environment and to provide a means of detecting changes to critical system files made by root-level adversaries. We discuss the design and implementation of the Forseti system, present a security analysis of Forseti, examine the performance of the system, and explore how Forseti can be used in concert with other security monitoring techniques to enhance the security of the HPC cluster environment.