Image
How Is YARA used?
Historically, common detection methods have used file hashes (MD5, SHA1, and SHA256)—unique signatures based on the entire contents of the file—to identify malware. Modern threat actors have increased in sophistication to a point where every instance of a given malware will have a different hash, and that hash will vary from machine to machine. The threat actors are disguising malware by making small (functionally meaningless) changes that result in a new hash, thus rendering MD5, SHA1, and SHA256 indicators of compromise ineffective. In 2015, Victor Alvarez created YARA, a pattern-matching Swiss army knife for malware researchers. YARA allows for improved description of malware families using textual or binary patterns. Each description (aka rule) consists of a set of strings or regular expressions and a Boolean condition to determine how many of each string or regular expression must match for the rule to fire. These rules identify critical functional sections within malware files. Once a YARA rule has been created, it is much harder for threat actors to alter or disguise malware to avoid detection by YARA. However, there is a downside to YARA’s complexity: full system scans may require a large amount of computing resources.Image
What’s Different about Osquery?
Around 2013, Facebook was challenged with running a large number of different endpoint agents, all of which added to a high endpoint resource utilization and none of which provided the desired level of telemetry. As a result, Facebook wrote its own endpoint agent. Thus osquery was born. Osquery is lightweight, read-only agent that provides a wide range of telemetry on Mac, Linux, and Windows. Osquery can give us details of each and every process running on a machine. It can also tell us all the network sockets opened, user logons, files that were changed or read, launchd & cron entries, firewall, configuration, and much more. As of Feb 2020, there are 236 different tables worth of telemetry available in the open-source version of osquery. And, you guessed it, osquery can run YARA, too!Why Combine YARA + Osquery?
Combining these two tools allows us to take advantage of osquery’s ability to identify which processes are running and to do a targeted YARA scan of just those files. Similarly, we can use osquery’s file monitoring capability to tell us when high-risk files are created or changed and do a targeted YARA scan of those files. With osquery and YARA together, we can still use the power and complexity of YARA, but we can reduce the required compute resources by targeting scans with telemetry from osquery. In one test, we scanned the major system directories (/lib, /etc, /usr, /var, /home) of our Linux test server and measured a 167 second total scan time with 48.6% CPU usage. Full system scans can be too resource-intensive for many IT shops. A better solution is to target scans based on triaged approach. We then ran a targeted YARA scan with osquery while monitoring resources.Image
Image
