I talk about patching almost every day. I talk to customers, I talk to peers, I talk to sales folks. I even talk to my kids about it – they kind of drift off when that topic comes up, but I still talk about it. Why do I spend so much time talking about something so trivial and boring? Because it is just about the most important thing you can do when it comes to securing your environment. If you don’t have a solid handle on the top 5 CIS controls (https://www.cisecurity.org/controls/cis-controls-list/), bad things are going to happen. When you are the last one leaving the office at the end of the day, do you make sure the door is locked? Of course you do. If you don’t have an effective patch management program, locking the door doesn’t really matter much. Sure, it will keep some of the folks out but not the ones you truly want to keep out.
No one likes to patch. I have never met anyone in all the years I have been doing this who said, “I CAN’T WAIT for tomorrow – it’s Patch Tuesday!” Application owners don’t like to test. End users don’t like to test. No one likes giving up nights and weekends to deploy the patches or deal with the fallout from systems that don’t come back online when you do patch them. No company wants to deal with the fallout when a production line goes down because you patched it. I’ve lost track of all the nights, weekends, dates with the wife and ballgames with the kids that I have lost over the years due to patching. What I always hate worse, however, is dealing with the drama when the bad guys take advantage of gaps. Those escalation calls NEVER happen at 10am on a weekday. They always seem to happen at 7pm on a Friday night and chew up my entire weekend. On my way out to dinner with my wife and my phone rings. I give her that look and walk back into the house – she orders pizza. Triage, forensics, incident response, remediation, etc. That entire conversation is so much worse than dealing with patching and hardening. And so much more expensive. Exponentially more expensive.
So, dealing with patching is a pain for the people who are actually responsible for doing it. It is also a bigger pain for the folks that have to look at compliance across the environment. Think of the security team that needs to figure out how compliant you are across all of the operating systems in the environment – lots of different tools, lots of different operating systems, lots of different maintenance windows and patching cycles – lots of drama there. Each tool with its own view of its portion of the world. There are better tools out there by the way.
Now, take all of that pain and make it worse. Not only do you have to patch on a constant basis, you have to harden your systems – and keep them that way on a constant basis. Having a system fully patched but using all the default out of the box security settings is almost as bad as not patching it at all. Regulatory controls in the IT space exist for a reason – CIS, DISA STIG, PCI DSS, etc. are there to tell you how to help keep your environment safe.
Mature environments that are really good at keeping their environment secure always know the status of their security. When someone spins up a new server or a laptop comes out of a filing cabinet after 6 months, it should get patched automatically regardless of the operating system. Mature environments can run a report anytime and tell you which machines or business units are the worst offenders. They know when things unexpectedly drift out of compliance. Those mature environments also have a hammer to swing to enforce compliance. Those are the customers that go out of their way to keep their names out of the newspaper and to avoid explaining non-compliance fines to the board. Those environments also assess, validate and verify on a constant basis. They have a variety of tools at their disposal, and they use them. They also bring in outside eyes regularly to validate what they think they know. Penetration tests, physical security tests, testing to see how gullible their end users are, how patched are their systems, how hardened are the systems. They also know that they can trust the data their tools are providing.
Do you know which companies are great at patching and hardening? It’s normally the ones you don’t hear about. Sure, breaches can happen regardless of patching and hardening efforts, but it is very rare to find someone that got successfully attacked or breached that you can’t point back to less-than-effective patching and hardening programs. I hate to play this card, but how many folks got hit with WannaCry or any later attacks leveraging the same holes? Do you know that I still have customers that are hit with it on a regular basis? Why?
If you are responsible for IT security, and you are still missing MS17-010, I can’t wait to hear why it doesn’t matter. Boards tend to not understand technical nuances, but it is hard to find a board member that hasn’t heard of WannaCry and hasn’t done at least some mental math to figure the impact to the company if a major outbreak happens. Keeping that conversation off the table is why we patch and why we harden. It’s why we pay attention to the CIS controls, especially the top 5.
I love to quote Reagan – trust but verify. All day, every day. Most tools out there in the patching space don’t really tell you the truth, or if they do, it is slanted – yes, I can tell you that there are no systems missing MS17-010. That is how the tool sees the environment. The tool omits the fact that it doesn’t know how many systems it is not reporting against (rouge IT, shadow IT, broken components of the OS, not in the domain, etc.). So, looking at that same truth through a different and more accurate lens might tell you that yes 10,000 systems have MS17-010 installed, but there are also 1,000 systems not managed by that tool that should be, and there are another 2,000 that have broken/corrupted components that those tools rely upon. So in reality, you might have another 10-20% of your environment that is at risk, and no one is telling you. Your trusted tools are not telling you. Trust but verify. How would you feel after putting forth all the efforts and making all the sacrifices you or your team does for patching and hardening only to find out you missed a noticeable chunk of the actual environment? How is that conversation going to go over after the bad guys find the gaps?
So, what do you do? You keep putting up the good fight – patch and harden, rinse and repeat, week in and week out. You make sure that your patching and hardening programs are effective. You make sure that you can trust your tools. You also put a separate set of eyes on the environment on a regular basis. Let someone that doesn’t have personal skin in the game have a look around. Even if it is just a portion of the environment, have an outside assessment performed. It is better to get the full picture, but some outside validation is better than none. Get some peace of mind or at least find out where you need to put some additional focus. Head in the sand is never the right choice in this space.
CBI can help you to keep your environment patched and hardened. Request a Patch Assessment today to get started.