I often ask technical evaluators and managers what they look for when they chose a vulnerability management solution. It is fascinating to observe how the answer varies with the maturity of the company’s vulnerability management program.
The Early Stages
In the very early stages of the vulnerability management deployment many respondents focus on features related to Ease-of-Use and Ease-of-Deployment. For example, at the top of the list there are usually references to hardware and software appliances, web-based user interfaces, reporting and configuration.
There is an overall expectation among new users that, as long as the vulnerability management team can complete the deployment on time, on budget and as per the design the project will be successful. There are elements of optimism and pressure from the business to leverage the initial investment in the technology.
A few months after going live, especially in very large deployments, disappointment and frustration sets in. The vulnerability management team, their managers and users have discovered by now that many of their initial assumptions were wrong or irrelevant to the task at hand.
The teams are now earnestly resetting expectations and working around the clock to try to get the solution to generate the specific vulnerability management reports required by IT operations and management. Some vulnerability management vendors provide flexible reporting required by the users.
However, large organisations often have to invest significant resources to export data into structured databases and build a reporting portal on top of the exported data. In another post I will cover the challenges and complexity of reporting solutions developed in-house.
Another source of frustration in the early stages of a new vulnerability management deployment is the lack of credibility the vulnerability management solution has with IT operations.
The vulnerability team is often quickly overloaded because they need to run manual checks to validate the discovery of assets not present in the official register and there is some work required to validate potential false positives and false negatives.
For a few weeks, the vulnerability team is often asked to demonstrate that their scanning data is accurate. Poor accuracy and limited coverage are significant obstacles that can lead to project failure, especially if IT Operations doesn’t agree to adopt the new reports.
After around 9-12 months, the vulnerability management solution has settled into the IT processes of an organisation. This is a good time to ask about the value that a vulnerability management solution provides to the business.
If the deployment is relatively successful, users will respond to questions about their choice of solution by discussing how it fits into their change control workflow or their IT Processes. They are no longer talking about technology or the value it delivers to their business.
Instead, they describe how the solution helps IT operations prioritise their daily routine, including patching and vulnerability mitigation.
On the other hand, if I ask someone whose project has not been fully successful for their feedback about their choices, they usually discuss several potential solutions to their ‘problem’. For example, they might consider outsourcing vulnerability management or focusing on a patch-driven approach.
At this stage, one interesting way to test how successful the deployment has been is to ask whether they are using credentials to scan. The use of credentials gives the scanner deeper visibility, including the ability to scan desktop applications or to detect vulnerabilities that traditional port scans cannot detect.
Most successful deployments are using credentials at this stage because IT Operations has seen the benefits of a vulnerability management programme. In contrast, less successful deployments are unlikely to be leveraging credentials many years after the initial deployment.
The most interesting conversations I have had with vulnerability management users are connected with who should have visibility into the scan data. In new deployments, there is an on-going tension between keeping some data confidential and disclosing it, usually on a need-to-know basis.
The impact of disclosing this level of detail to the rest of the organisation is rarely clear to new vulnerability management users.
This response stands in stark contrast to successful, mature deployments. In some of the best deployments I have seen, vulnerability data is openly disclosed to the rest of the organisation.
Teams organized by business units, subsidiaries or geographical locations compete against in each other to reduce the vulnerability risk of their assets. Some organisations have taken this approach one step further by aligning bonuses and MBOs with specific risk reduction targets.
Over time, technology vendors also mature and change focus. Some have discontinued or stopped investing in their vulnerability scanners, forcing customers to look for a new vendor. The new solution is unlikely to be a one to one replacement for the previous solution or a perfect fit for the existing remediation processes.
As a result, the organisation has to build new processes or adjust existing ones.
Image courtesy of ShutterStock