As a field consultant for a security software company, I get the opportunity to work with a wide range of customers with an even more varied collection of issues. I am called in for all sorts of reasons, and sadly, many of them are not the most stellar. One of the more difficult comes when investigating what the customer believes is a small issue, but it quickly grows, seemingly out of control, as more and more is discovered.
There is a term in my house for issues such as these: A Beeswax Seal incident. The origin of the phrase trace back to the experiences of a family friend. It started with a small, afternoon project to replace an aged beeswax seal on toilet and ended up as a multiple-month second-floor remodel including the bathroom, the second-floor landing, a child’s bedroom and some structural updates supporting the aforementioned rooms.
Each time he took one step in the process, he uncovered another issue that had to be addressed. The cause, or effect, of the condition he was attempting to repair generated another set of conditions, another solution that had to be factored in, and several more trips to the hardware store.
This condition is not limited to home-owners and other construction projects, many of the things we need to fix have the potential to lead us down the same path of discovery and scope-creep. In my work as one of Tripwire’s Security Software Consultants, I visit many customers who may be facing just that. I’m there to get our tools in, to help them take a look and see what is really going on.
When dealing with a Beeswax seal incident, we have to keep a few things in mind. Thankfully, what works for those, works for other areas as well.
To quote one of my favorite authors, the first order of business on any creeping issue is “Don’t Panic”. We need to accept and understand that the project will have to change, and if we are lucky, the person that designed the plan allowed us some room to adjust.
Remember, either we are working with a project team, or doing a DYI project at home, in this internet-connected world, we are not alone. If there is any uncertainty about the next step, we know that there are people to ask. Experts exist in just about every subject. They are just a webpage, an email, or a phone call away. Even a quick decision does not need to be a rash, unconsidered one.
In fact, many times the consequences of a poor response can sometimes be greater than the problem itself.
Next, we want to isolate the issue without worrying about the cause. Turn off the water to stop a leaky faucet. Consider unplugging a network cable to isolate a compromised system. Remember that we want to be able to understand what happened in order to keep it from happening again. So many people find it easier to system wipe and restart, or replace the beeswax seal and be done with it.
We need to be able to take the time and look around. We can’t fix things properly if the response is to just cover the crack up with a little paint.
With the immediate fire fought, allow some time to do a little investigation. Ask some questions about the state of the issue and make sure we know what happened, how it happened, and why it happened. Why are we preforming a DoS attack on our own firewall? How did the system get compromised? Why does the sub-floor look funny and squeak when I step here?
This is where proper tools for the job come in. What building inspector does not have a flashlight? Without it, they would not be able to look into the darkest corners of the house and help track down the root cause of the issues they are seeing.
In the security world, we use the SANS 20 Critical Security Controls for guidance on data we need to collect and for protections we should implement. File integrity monitoring, a central log collection and vulnerability assessment are all examples of tools that should be in any security team’s “toolbox” to act as our flashlight.
Data in hand, we should be getting a much better idea of where things stand. And I’m sorry; I know there are going to be some realities that people will not want to face. So many times, the results are not pretty. But bad news is not like fine wine; it is not going to get better with age.
So we face the news and start making our adjustments. Shopping lists are written up and purchased. Experts are re-consulted. Bosses / spouses are informed. In some cases, it’s going to be painful, or embarrassing, but it needs to happen. Also, don’t be afraid to call in an expert. “DYI electricians are in for a shocking discovery.”
If there is limited experience on the immediate team, make sure to have someone with experience double-check. It is a better long-term play.
As repairs are proceeding, factor in some watch-guards in key places. If there were problems with data collection, take care to solve those oversights. Put a moisture sensor in the floor to sound an alarm if there is another leak. Add some entries to the SIEM and Configuration Assessment tools to alert the team if the intrusion’s fingerprints show up again. Now is the time to get an ounce of prevention in place so we don’t have to pay for another pound of cure later.
So, everything is back in place and the worse is over. The server is back up and running and the network is clear, or the family is all moved back in and the daughter loves her new room. But the job is not over. In fact, it never will be. “Constant Vigilance” is now the term.
But what this experience has given us is a first-hand understanding of how things work that we may not have had prior to the project. And better yet, we would know some warning signs to keep an eye out for. Be it an odd smell and a floor that creaks when it shouldn’t, or unauthorized file changes and abnormal network traffic. They are now all things that are instinctively known to need future investigation.
Plus, we already bought the tools, so why not put them to work?
- Leveraging the Windows Registry in Digital Forensics Investigations
- Digital Forensics and Incident Response
- Tales From the Crypto: Case of the Malicious IT Contractor
- Philip Polstra Discusses Digital Forensics
The Executive’s Guide to the Top 20 Critical Security Controls
Tripwire has compiled an e-book, titled The Executive’s Guide to the Top 20 Critical Security Controls: Key Takeaways and Improvement Opportunities, which is available for download [registration form required].
Title image courtesy of ShutterStock