Skip to content ↓ | Skip to navigation ↓

This week I’m skipping an entry in my security automation series, and instead posting about what I’m doing right now: Attending the 7th Annual IT Security Automation Conference near Washington, D.C.  In addition to gaining a substantial degree of insight into the future of Security Automation and (the new buzzword of the day) Continuous Monitoring, I had the honor of participating in a vendor panel on Continuous Monitoring at the 7th Annual IT Security Automation Conference.

I was surprised that we could only address two of the moderator-supplied questions – the audience took over, which is the best kind of panel to be on!  The feeling I came away with is that Continuous Monitoring is truly desired in the federal space – it’s something that has been proven to be effective in three separate agency environments.  However, I also had the distinct feeling that many of the vendors were communicating that they already do “continuous monitoring,” and they’re right.  (I explain what I see as the difference a little later).

Other participants in the panel included Kent Landfield (McAfee), Tim (TK) Keanini (nCircle), and Kathleen Moriarity (EMC).  It was an enjoyable experience and I learned a few things.

  • The questions the audience asked implied a great deal aobut thier honest interest in Continuous Monitoring.
  • The vendors on the panel all cautioned against believing that Continuous Monitoring is ready to roll – we have a long way yet to go.
  • TK made the excellent point that what Continuous Monitoring really needs at this point is some Product Management – this after I asked the question: How many in the audience are not federal employees, contractors supporting a federal contract, or vendors?  Three people raised their hand out of perhaps 80.

So, this is all well and good, but it’s time we look at what “Continuous Monitoring” really is in this context.  If you’re familiar with Boyd’s loop (ever hear of “plan, do, check, act” in ISO 27000?) then you’re looking at the basis for Continuous Monitoring in this context.  It’s the enabling of continuous compliance at the macro level.  Therefore, I like to reference Continuous Monitoring in that way, as Macro Continuous Monitoring.  This is aligned with the concepts presented in NIST IR 7756 (CAESARS FE; PDF) and NIST SP 800-137 (Continuous Monitoring for Federal Information Systems; PDF).  Compare this to the continuous monitoring that most vendors claim (Tripwire included).  This level of monitoring relates to what I call the Micro Continuous Monitoring – the monitoring of your specific control (i.e. File Integrity Monitoring) that essentially provides a feedback loop between check and act in the PDCA cycle.  Without distinguishing between Micro and Macro Continuous Monitoring, it’s easy to talk past each other.

Just for fun, here are some of the questions we heard, and my related answers (note that Continuous Monitoring is at the macro level and abbreviated as “CM”):

  • What are the drivers that led you to an interest in CM modeling and standardization? Continuous monitoring is the next logical step as we follow the path of automating security.  I see four drivers behind security automation, and in turn continuous monitoring: Increasing system complexity, threat agent motivations, speed of change in our environments – operational and threat, and a tangible scarcity of security practitioners.  These drivers require automation, and to take advantage of that which we can automate requires the basic capabilities of analyzing. It’s also not difficult to envision a world where stakeholders are able to interrogate the security health of a system at any point in time.  Consider or any other cloud provider.  Wouldn’t it be an interesting world if any user could run basic queries against the portion of that cloud affecting their information and services?  CM has the potential to enable that kind of capability and transparency.
  • Does the industry need standardization and reference models surrounding continuous monitoring? The need for standardization is quite clear, and we can look outside the security domain to find some justification.  How many of you have heard of Generally Accepted Accounting Principles?  Briefly, GAAP has evolved to provide a common vocabulary when organizations report financial results, wether for regulatory or investment purposes.  We need to arrive at a similar point in the security domain – so we can compare apples-to-apples and make decisions based on real data.  Hopefully, it won’t take us more than 70 years to arrive at a GAAP for the security domain.
  • What are the primary challenges that will exist for vendors in adopting the presented model? Two things.  First, it’s going to be interesting to see how each of the described subsystems will be embodied in products.  In theory, a single product can do all of the CM tasks.  In reality, some vendors will produce hybrids of various CM subsystems and others may prefer to create products for each subsystem that can operate on a stand-alone basis.  Second, and perhaps more important, if customers aren’t demanding products that fit into this CM architecture, vendors will hardly be motivated. (This is when I asked the audience how many of them were in the commercial space and only three answered affirmatively.) Customers include commercial markets on a global scale.  The federal government has purchasing power, but I don’t believe it’s powerful enough to outweigh that of the global commercial market.  For vendor adoption to really take off, we need strong, international adoption of these standards.
  • From what you have seen, what are the biggest challenges that remain in finalizing the model? The number of remaining specifications and getting the relationship between those specifications right is a huge challenge.  My hope is that the CM working group makes a concerted effort to reuse specificaitons where it is warranted.  Workflow and rules languages come to mine here.  CM is moving forward, but, even though SCAP and it’s constituent parts have met some success, they have some way yet to go.  We run some degree of risk in ensuring the interoperability of the specifications, which has to happen before we can really have interoperability in the field, out of the box.
  • What hurdles will organizations face in composing multiple security products together to form CM solutions (both with and without standardization)? In a standards-based world, we will still have interoperability challenges until the specifications are, well, specific enough.  This will probably take vendor collaboration. It is also not difficult to imagine a world where different vendors have packaged CM subsystems in different ways.  I mentioned this in an answer to a previous question, and this is the other side of that same coin.  If vendors are packaging CM subsystems in a variety of ways, it’s not unreasonable to expect some overlap in capabilities.  This may be viewed as a good thing – the customers have choices. In a non-standards-based world, enterprises need to be concerned with all of this on their own, find a single-source provider of these capabilities, or rely on vendor alliances to fill the need.  This has worked in the past, but it doesn’t really get us to a point where we can offer queries up to any stakeholder at any time on any system or service. The reference model is still incomplete.  We have not yet started thinking about the analysis subsystem in any detail, much less how the interface to that subsystem should be specified.  The coupling of these two aspects are, for the time being, being left to the vendors. One aspect of CM that will be a challenge is actually achieving out of the box interoperability between vendors.  This exists today in the security automation space, even though there is a validation program – interoperability is difficult.

My intent in providing these questions and answers on this blog is to get non-government organizations thinking about whether they could use Macro Continuous Monitoring in their environments, to see how much interest there is in the commercial sector (hint: I think there should be much interest in the commercial space), and to start some level of discussion on how enabling some degree of Macro Continuous Monitoring can help reduce an organization’s overall compliance costs and help them step toward true risk management, thereby achieving some degree of reasonable security posture.