Fail2ban is a log scanner, that is commonly used to defend SSH servers from brute-force attacks. It does not stop an attempt before it happens but bans the IP address to prevent additional attempts. This mitigates brute force attacks which are essentially throwing a dictionary of passwords at the SSH server. By gaining access to the command line on a host hackers can get information saved on that system or turn the host into a location to attack other hosts or users of a site being hosted.
Fail2ban can aid in the mitigation of DOS, "Denial of Service," by blocking attacking IP addresses. However, if a denial of service is a distributed DOS or DDOS such an attack can overwhelm the network resources supplying the host; Nonetheless, the service running on the host may be overwhelmed far easier than the network leading to the host.
When potentially harmful attempts, such as a brute force attempt to discover a password, are made to a service. fail2ban goes into action based on the specified configured action. By default, this is to block the IP address from the service using iptables firewall for a period of time, which has been configured in the jails files.
Fail2ban does not clean the log files. Hackers can exploit the fact that some of the information in a log file comes from the user. These programs need to ensure the proper sanitation of this data. Many of the exploits used by hackers are to embed information in log files to escape from the program causing it to crash or to embed code into the log files which if not sanitized the log scanner could execute.
The bug in vRealize was essentially trusting data that was from virtual machines that could become compromised. vRealize reads log files to determine resource usage of virtual machines on the host. The bug was discovered by reviewing code. Trust but verify.
Fail2ban is in the repositories of virtually all Linux distributions. It is installed easily by the package manager and includes an assortment of filters that have been peer-reviewed and are ready to turn on. This method of installation maintains having the latest version when an apt-get update & apt-get upgrade is entered on the terminal screen. It is essential to insure all log scanners are running the latest version.
When a filter, (specified by jail configuration files), detects an action that should be banned jail configuration decides if an action should put the IP address in a jail. If a password is entered incorrectly it can be set to provide a fixed number of retries over a given time. The Jails section also determines how long an action will result in a ban.
Filters are essentially regex expressions that are compaired to log entries. When a filter detects a fail or activity that should be banned it generates an action.
The actions sections call other applications on the server such as the firewall to block the IP address from the service.