Cyber-attacks are continually increasing in scope and complexity; advanced persistent threats are becoming more difficult to detect; and over the past decade, there has been a growing “detection deficit,” according to the
2015 Verizon Data Breach Report. While 60 percent of attackers are able to gain access within minutes, the detection of attacks is usually days or longer. The core of this detection deficit is the fact that the cost, complexity and volume of data needing to be analyzed increases with the maturity of the security organization.
Many organizations are collecting log data from systems, applications and network devices to generate operational statistics and/or alerts to abnormal behavior. However, most of the valuable security data is hidden from view in such log data, due to the fact that software engineers write the code that determines what gets logged within their applications.
Unfortunately, a lot of valuable data is not written to logs, making it impossible for log management systems to detect attacks quickly. The best method to detect attacks is to analyze the session, packet string, and full packet capture data within the environment. The sheer volume of packet capture data that traverses a typical enterprise network is unfeasible to store for forensic purposes.
Bro is an open-source network security monitoring tool which has an extensible scripting engine for analyzing packet data. There is a wide array of pre-written scripts that ship with Bro out of the box that analyze network traffic, while giving the user the ability to create their own. These scripts analyze protocol, session and application layer traffic, looking for abnormalities and potential malicious behavior. Each finding is written to a local log file that can be collected and analyzed by a log management product, such as Tripwire Log Center
log management.
An open-source log management solution that is available for use is the ELK stack;
Elastic Search,
Logstash and
Kibana. Logstash can collect, normalize and send the log data to Elastic Search for long-term storage.
The power of Logstash comes with the filter plugin, which allows custom fields to be added, geolocating of IP addresses, and much more. Additionally, Kibana is a power visualization tool that can expose critical data quickly.
While collecting the default Bro log data can provide insights into the normal and abnormal behavior of the network, integrating threat intelligence can bubble critical events to the security operator's attention quickly.
Critical Stack Intel is a quick and easy way to get started with threat intelligence. The Critical Stack service provides more than 80 threat feeds containing nearly one million indicators of compromise. These IOC's can be imported directly into Bro scripts for real-time analysis.
With a couple clicks of the mouse and a single command line, Bro can begin looking for known malicious IP's, malicious file hashes and known phishing domains, among many others. Logstash integration with threat intelligence data feeds does not come as simple as the Critical Stack integration but there are easy options for integration. Using translations during normalization, we can compare the detected domains, IP's, hashes, etc. against indicators of compromise to enhance the data collected.
I will be speaking to these solutions in detail at Black Hat USA this year. Come join me in the
South Seas conference room at the Mandalay Bay on August 6 at 9 AM to learn how to use these tools to your advantage.
I will be diving deeper into how to customize Bro, Logstash, Elastic Search and Threat Intelligence to obtain security context with Kibana visualizations.
