The Internet Security Alliance (ISA) has released two more short papers which outline objective criteria by which President Obama’s Executive Order on cybersecurity should be evaluated in terms of its actual effectiveness in improving the nation’s cyber security.
The Administration is set to release the NIST Cybersecurity Framework initiated under the Executive Order on February 12, and ISA hopes this evaluation criteria can help frame policy makers’ analysis of the complex issue and help point toward needed next steps.
The first paper analyzes the failure of the NIST Framework to adequately address the cost-effectiveness criteria (Word Doc) called for in the President’s Executive Order, and the second paper suggests that assessing the success of the program will be difficult because there is lack of clarity (Word Doc) on a number of key issues including the overall goals of the effort.
The third paper calls for more work to be done on generating market incentives (Word Doc) to encourage participation in the Framework, noting that very little has been accomplished in the last year to incentivize voluntary adherence to the standard.
“Especially given the fact that the Framework development process itself failed to address the required cost effectiveness issue, the lack of progress in developing the incentive proposals industry has made has resulted in a Framework of indeterminate cost and benefit with no added motivation for its voluntary adoption. Thus, it is questionable as to if the framework effort will result in any significant security beyond what already exists,” according the ISA report.
The fourth paper the ISA released asserts that the value and impact of the Framework should be evaluated using a private sector model (Word Doc), and not left to subjective assessments, citing “growing support for its proposal to have the effectiveness of the cyber Framework determined by systematically gathered data from defined target audiences.”
“Obviously there a wide range of items people can look at and make judgments about the polices outlined in the Executive Order, but cyber security is too important to be left to impressions and opinions, we need to use the cyber Framework to generate hard data about what works and then promote those policies that make sense based on the data,” said ISA President Larry Clinton.
“Tthe NIST Framework has not been sufficiently tested in order to merit broad scale deployment and implementation at this time. There has been no baseline data developed against which to measure the cost or effectiveness,” the ISA stated.”
“Moreover, there is no data presented as to the actual effectiveness (i.e., impact) of the standards and practices suggested by the Framework. Put simply, there has been no data that would assure an entity potentially interested in adopting the Framework that by doing so it would be any more secure, let alone secure in a cost effective manner.”