What’s the Point of Compliance?The consultancy, certification and advisory industry make a profit by selling compliance. They have to tread a fine line between helping organisations to improve their security measures and placating them. Why does this tension exist? Because it’s not uncommon for organisations to fire consultants they find to be too hard. Consultants are put under pressure to let things slide. If paying your bills is dependent on letting things slide for an organisation, some consultants will do just that. Whilst there are controls to manage this risk, independent audits, reviews etc., none of them are perfect. I am by no means saying that all consultants are prepared to sign anyone off who pays. However, I absolutely come across organisations that have been marked as compliant, that during an audit are found to be so far from compliance that it takes my breath away. These organisations have either lied to the consultants (which any decent consultant should find through checks), or the consultant has lied. The problem is ongoing. To get compliant costs money, to remain compliant costs money, and money introduces conflicting motives. They also, in some cases, effectively operate as a cartel—for example, under the PCI standard, you need to use ASV (approved scan vendor) scanning, which includes a limited list of approved vendors, all of whom have paid money to be on the list and proven their scans meet the standard. They effectively get a large market that can only go to a limited number of suppliers, like shooting fish in a barrel.
Why Doesn’t Compliance Equal Security?Compliance as an annual process is an interesting idea—a year is a reasonable period to rectify issues. However, even with an external, independent (reputable) auditor, an organisation’s actual compliance status cannot be guaranteed at any time apart from the day the organisation successfully passed its last audit. If you consider the number of changes a large organisation makes on a weekly basis, the fact it was audited six months ago, cannot possibly indicate that today it is definitely still compliant with the standard it was audited against. This issue can absolutely be mitigated through conscientious examining and tracking of standards. Nonetheless, the reality of compliance being a static measure of a dynamic situation highlights the folly of taking a compliance certificate as an indication of current compliance. Despite the pervasiveness of accepting compliance certificates as an accurate indication of an organisation’s compliance status, the practice carries real risk. The standards of consultants are variable. It is not uncommon to find graduates with a few years’ experience being used as auditors and signing organisations off. As audit is a profitable business for consultancy organisations, getting recruits and training them to operate to the standard (which is relatively basic), means we have cheap consultants being sold expensively and operating at the most basic level. They might well get you through the standard but the customer often knows more, and the consultant’s objective is to get you signed off and certified and move onto the next job. The more in-depth consultants will want to make absolutely sure you meet the standard and will continue to do so. They will want to understand at a very technical level what you do and how you do it, while the really good consultants will want to make damn sure you exceed the standard.
The Problem With StandardsMost compliance standards also allow for the scope of the compliance to be limited. For example, PCI has the CDE (Card Data Environment) and ISO 27001 has the scope of compliance. Effectively, you are able to delimit the area of compliance. Again, this becomes about demonstrating compliance to third-parties, not about securing the full environment. PCI is an excellent example of limiting the scope of security measures to a narrowly defined area. PCI is only concerned with securing card data, it doesn’t consider anything else (this makes sense when you consider the card industry wrote the standard). The problem is that organisations are using PCI compliance as an indication that they are secure but it doesn’t mean that; it can’t. PCI compliance only indicates that the card data handled and stored by an organisation is being secured. The rest of the customer’s data held by an organisation is not accounted for. This represents a disconnect from reality. Whilst for PCI card data should be stored in a more secure CDE, if an organisation’s less important networks can be hacked, then hackers in time can work their way into the CDE. Standards do not reward for over compliance. Most standards are simply a pass or fail – you’re either compliant or you’re not. This means that organisations will aim for the lowest possible standard, i.e scrape through. Given that standards are designed to be achievable, (if they are too difficult people will not comply or lie, standards have to be achievable). So, standards are the lowest common denominator. Effectively, you end up with a weak standard. There is an argument to be had that a weak achievable standard is better than a more aggressive standard that organisations will give up on. However, my opinion is that this waters down standards too far. Perhaps it’s time for grades in compliance. Compliance is often seen as an attempt to offer a return on investments in security. Effectively, you are getting something for the money you spend on security – a certificate or logo to go on the website – it is understandable that the board want something in return for the money spent, but what they are getting is compliance with a standard, not necessarily an improvement in security. It can be argued that spending the money on pure security would produce better security, but it’s very hard to demonstrate this benefit to stakeholders. Investors, other organisations and regulators all want tangible value for their investment.
The VerdictAm I saying compliance is bad? No, absolutely not. What I am saying, is that compliance for compliance's sake does not automatically improve security in a meaningful way. However, security to improve security, does improve security and can have compliance as a framework. It is also absolutely true that compliance can be a framework to hang security off and help people understand it, but on its own, compliance won’t achieve much. As a security consultant who has worked across most industries, I have seen compliance done for the sake of compliance, as well as compliance done to make things better as part of security. When it’s done on its own, it is an indication that security is not on the radar, and that’s a real worry, even if an organisation becomes compliant.