How Is YARA used?
Historically, common detection methods have used file hashes (MD5, SHA1, and SHA256)—unique signatures based on the entire contents of the file—to identify malware. Modern threat actors have increased in sophistication to a point where every instance of a given malware will have a different hash, and that hash will vary from machine to machine. The threat actors are disguising malware by making small (functionally meaningless) changes that result in a new hash, thus rendering MD5, SHA1, and SHA256 indicators of compromise ineffective. In 2015, Victor Alvarez created YARA, a pattern-matching Swiss army knife for malware researchers. YARA allows for improved description of malware families using textual or binary patterns. Each description (aka rule) consists of a set of strings or regular expressions and a Boolean condition to determine how many of each string or regular expression must match for the rule to fire. These rules identify critical functional sections within malware files. Once a YARA rule has been created, it is much harder for threat actors to alter or disguise malware to avoid detection by YARA. However, there is a downside to YARA’s complexity: full system scans may require a large amount of computing resources.
What’s Different about Osquery?
Around 2013, Facebook was challenged with running a large number of different endpoint agents, all of which added to a high endpoint resource utilization and none of which provided the desired level of telemetry. As a result, Facebook wrote its own endpoint agent. Thus osquery was born. Osquery is lightweight, read-only agent that provides a wide range of telemetry on Mac, Linux, and Windows. Osquery can give us details of each and every process running on a machine. It can also tell us all the network sockets opened, user logons, files that were changed or read, launchd & cron entries, firewall, configuration, and much more. As of Feb 2020, there are 236 different tables worth of telemetry available in the open-source version of osquery. And, you guessed it, osquery can run YARA, too!
Why Combine YARA + Osquery?
Combining these two tools allows us to take advantage of osquery’s ability to identify which processes are running and to do a targeted YARA scan of just those files. Similarly, we can use osquery’s file monitoring capability to tell us when high-risk files are created or changed and do a targeted YARA scan of those files. With osquery and YARA together, we can still use the power and complexity of YARA, but we can reduce the required compute resources by targeting scans with telemetry from osquery. In one test, we scanned the major system directories (/lib, /etc, /usr, /var, /home) of our Linux test server and measured a 167 second total scan time with 48.6% CPU usage. Full system scans can be too resource-intensive for many IT shops. A better solution is to target scans based on triaged approach. We then ran a targeted YARA scan with osquery while monitoring resources.
In contrast with the full file system scan that took a 167-second total scan time with 48.6% CPU usage, the targeted YARA scan occupied only 4.0% CPU core and took only a second. For all the details on how to use these two great open source tools (YARA and osquery) to help secure your IT environment in a resource-efficient manner, join my talk at BSides San Francisco on Feb 22, 2020: https://sched.co/YbkM or listen to the recording available after the talk.
About the Author: Julian Wayte is an IT security professional with 20 years’ experience. Julian graduated from the University of Western Australia with degrees in Mechanical Engineering and Computer Science. His early work was in Data Warehousing, CRM, and Identity Resolution before moving to IBM to work on Data Encryption and Database Activity Monitoring. In 2019 Julian fell in love with osquery and started with Uptycs. Editor’s Note: The opinions expressed in this guest author article are solely those of the contributor, and do not necessarily reflect those of Tripwire, Inc.
Meet Fortra™ Your Cybersecurity Ally™
Fortra is creating a simpler, stronger, and more straightforward future for cybersecurity by offering a portfolio of integrated and scalable solutions. Learn more about how Fortra’s portfolio of solutions can benefit your business.
