Skip to content ↓ | Skip to navigation ↓

The Internet Security Alliance (ISA) has released the first two in a series of short papers which outline objective criteria by which President Obama’s Executive Order on cybersecurity should be evaluated in terms of its actual effectiveness in improving the nation’s cyber security.

The Administration is set to release the NIST Cybersecurity Framework initiated under the Executive Order on February 12, and ISA hopes their criteria, which will be presented in the form of five simple questions each with a short explanatory paper, can help frame policy makers’ analysis of the complex issue and help point toward needed next steps.

The first paper analyzes the failure of the NIST Framework to adequately address the cost-effectiveness criteria (Word Doc) called for in the President’s Executive Order, and the second paper suggests that assessing the success of the program will be difficult because there is lack of clarity (Word Doc) on a number of key issues including the overall goals of the effort, who the NIST framework is supposed to apply to, and how exactly enterprises are supposed to interpret the framework in terms of their own business.

“There is virtually nothing new in the framework. Moreover, there are no criteria to indicate how the adoption of various standards and practices will improve security, and although the President’s Order required to the framework to be cost effective, there is almost no analysis of this critical issue in the framework documents. If the goal is to have industry adopt the framework on a voluntary basis, its cost effectiveness is an essential element,” said Clinton.

The ISA report cites several studies that indicate cost is the single biggest problem in security critical infrastructure from cyber attacks and points out that if the Framework was demonstrated to be cost effective, as required by the Order, then its voluntary adoption could be expected on the part of most, if not all, reasonable owners and operators.

“To know if we are succeeding we have to know what counts as success. Increased awareness? Numbers who use the framework? Reduction in attacks or losses?…. Lack of clarity on what counts as success can be dangerous by leading to a false sense of security,” The ISA report stated.

“Policy makers need to understand that using the framework is not the same thing as assuring critical infrastructure security—much more is needed. Such a misunderstanding could lead to misguided public policies. Similarly, senior executives need to be clear that funding IT budgets up to a level simply adequate to adopt or use the framework, may not protect them from sophisticated attacks.”

ISA also questions the assertion that the Framework provides a language for senior executives to discuss cyber security with their IT teams, particularly in smaller companies, saying that the most recent draft seems predominantly geared to the technical and Executives may find discussions generated by publicity about the framework very frustrating.

“Most senior executives will ask, ‘have we in adopted or are we in compliance with the Framework?’ When told it is impossible to answer these questions clearly, and that the goal is to simply use the Framework,” the ISA report says.

“A logical follow up may be, ‘Well, do we use it?’ The likely answer being that we do use some of it.  Executives may also be unsatisfied with vagueness as to where they should spend their limited budget for most effect or if they are reducing their liability by using the Framework.”

Related: ISA’s Larry Clinton on Incentivizing the Cyber Security Framework