Image
- Hail a TAXII – This is a freely usable TAXII server that is publicly accessible on the internet, put up by our friends at Soltra. It has some open source threat intelligence on it, and that makes it a great place to connect to pull a TAXII feed from. It is not a test-bed though, so it may not be a great place to test non-compliant experimental code.
- Soltra Edge – Soltra, one of Tripwire’s partners, has a freely available TAXII server you can download as a virtual machine and easily stand-up. This would be the option I’d recommend, as you can have it on a virtualization server or on your local system using something like VMWare Workstation, and start to create TAXII feeds and test your code with it.
# These two modules handle the HTTP(S) protocol for us
import urllib2
from urlparse import urlparse
# We'll use ElementTree to parse the XML as it's a simple binding which ships with Python
import xml.etree.ElementTree as ET
# These are used to help with HTTP basic auth and generate random message IDs, respectively
import base64
import random
# The namespace of the TAXII document we'll send to the server
taxii_ns = 'http://taxii.mitre.org/messages/taxii_xml_binding-1.1'
# This class parses the CybOX results from the server
class CybOXObject(object):
# xml parameter should be an ElementTree.Element with the CybOX object inside it
# url parameter is the feed location
def __init__(self, xml, baseurl):
# construct a new URL we can use to browse to this object in the Soltra dashboard
p = urlparse(baseurl)
self.url = '%s://%s/object/%s' % (p.scheme, p.netloc, xml.get('id'))
# Find all file hashes using the ElementTree "findall" method. Each
# Simple_Hash_Value XML node can contain multiple hashes separated by a delimiter.
self.hashes = []
for hashval in xml.findall('.//{http://cybox.mitre.org/common-2}Simple_Hash_Value'):
self.hashes.extend(hashval.text.split(hashval.get('delimiter', '##comma##')))
# Internal function constructing an HTTP request to read unread threat data via TAXII
def _taxii_req(url, feedname, username, password):
# Construct an XML document with 'Poll_Request' as the top-level node name.
xmldata = ET.Element('Poll_Request', {'xmlns': taxii_ns,
'message_id': str(random.randint(345271,9999999999)),
'collection_name': feedname})
# Fill in a few TAXII parameters
pp = ET.SubElement(xmldata, 'Poll_Parameters', {'allow_asynch': 'false'})
ET.SubElement(pp, 'Response_Type').text = 'FULL'
ET.SubElement(pp, 'Content_Binding', {'binding_id': 'urn:stix.mitre.org:xml:1.1.1'})
# Create an HTTP request with our XML as the POST data
req = urllib2.Request(url, ET.tostring(xmldata))
# Add some headers to the request and return it
if username is not None and password is not None:
req.add_header('Authorization', 'Basic %s' %
base64.encodestring('%s:%s' %
(username, password)).replace('\n', ''))
req.add_header('Content-Type', 'application/xml')
req.add_header('User-Agent', 'TAXII Client Application')
req.add_header('Accept', 'application/xml')
req.add_header('X-TAXII-Accept', 'urn:taxii.mitre.org:message:xml:1.1')
req.add_header('X-TAXII-Content-Type', 'urn:taxii.mitre.org:message:xml:1.1')
req.add_header('X-TAXII-Protocol', 'urn:taxii.mitre.org:protocol:https:1.0')
return req
# Function to poll a TAXII feed. Returns a list of CybOXObject instances
def poll_feed(url, feedname, username, password):
return [CybOXObject(x, url) for x in
ET.fromstring(urllib2.urlopen(_taxii_req(url, feedname, username, password)).read())
.findall('.//{http://cybox.mitre.org/cybox-2}Object')]
# Example command-line usage
if __name__ == '__main__':
import sys
if len(sys.argv) < 3: print "usage: taxii.py url feedname [username] [password]" sys.exit(-1) url = sys.argv[1] feedname = sys.argv[2] username, password = None, None if len(sys.argv) >= 5:
username = sys.argv[3]
password = sys.argv[4]
for obj in poll_feed(url, feedname, username, password):
if obj.hashes:
print "%s: %s" % (obj.url, obj.hashes)
Source code examples are released under the BSD 2 Clause License found at http://opensource.org/licenses/BSD-2-Clause.
As you can see, this is actually pretty straight-forward. Browsing through the relevant standards, one might think that STIX and TAXII require months of study to successfully consume. But most of the complexity at the XML level lies in CybOX, the dictionary for which needs to be voluminous to handle all the different possible types of threat data. But if your organization is only interested in a few kinds of data, it's easy enough to pick out and handle the elements you want (as the example above does with Simple_Hash_Value) and ignore the rest. In fact, STIX, which acts as a container for CybOX, is never addressed at all here; instead we just use the ".//" syntax in XPath to skip right past it to the actual content.
Likewise, while it's possible to create complex feed scenarios with TAXII, starting out as a consuming client is virtually as simple as sending any other HTTP request; in the example above, we constructed a poll request in four lines of code, then added a few TAXII-specific HTTP headers. Users can extend this basic functionality to handle requesting indicators over a range of timestamps, etc., but for the purposes of discovering any threats which have come in since the last poll request (which Soltra supports by default), the code shown here is pretty much all you'll need.
So, have you tried consuming external threat data using STIX and TAXII yet? If you were waiting because the setup seemed too complex, I guess you'll need to find another excuse.
Resources:
Image
