Niente panico! È solo un incidente informatico

Good afternoon. Happy Thursday. Welcome to our webinar today. My name is Dustin Sachs. I am Chief Technologist and Senior Director of Programs, the Cyber Risk Collaborative, which is the membership community of CISOs here at Cyber Risk Alliance. I'm really excited to be here today with with Jim Doggett, who is going to be talking with me about our topic today, which is Don't Panic! It's just a cyber incident. This month at Cyber Risk Alliance and at the Cyber Risk Collaborative, we're talking about business continuity, disaster recovery, incident response. And this topic is really gonna be exciting and interesting. We're gonna talk a lot about, like, what does it mean to actually be in an incident. Before we get started, I wanna thank our sponsor for this month, Semperis. We're very fortunate to have our sponsors who help us do these types of events. So to start off with, I'm gonna let Jim introduce himself, and then I'll give a little bit about my background and why both of us kind of belong in this conversation. So, Jim, great to see you today. You too, Dustin. I appreciate, inviting me today. Fantastic topic, to me especially for those in the CISO realm. Historically speaking, we had a little bit involvement, but with ransomware today and the attacks, security is causing or gaps in security are causing actual incidents to occur. So we've gotta be a little more attuned to it today. My background, been doing this a very long time. Was with for twenty seven so or so years. Half of that is an auditor and half of that is a security consultant. And from there, I, moved on and would spend some time at JPMorgan, AIG, and Kaiser Permanente, couple different industries, but all in a CISO type role in a sort of a risk officer, IT risk officer role. So a lot of experience in doing this. Been through a fair number of incidents. No catastrophic ones, but this is certainly something I've spent a good bit of my career thinking about and then talking about. Yeah. Absolutely. And as our audience knows, but for those who don't, and I think I shared this in our pre call. You know, I come from twenty years of industry. I started out in computer forensics, worked on incident response, worked on some very large incident responses of retail and education and, you know, I mean, you name it industries. I came in as an outside consultant to these incidents. So I was the person coming in in the first couple hours after the incident and then investigating it and helping to lead those investigations. So I think our topic today is gonna be really good, and I think we're both gonna have a fair amount of, war stories and experiences from this. So, you know, when we talk about incident response, you know, everybody sees the stories in the news. We hear about this stuff. But I wanna start off by talking, and and I want us to share kinda share with the audience what that first few hours looks like. You know, you get the call or you're you know, if you're the CISO, you get a call from somebody on your security staff. Hey. We think we've got an incident. We think we've we've been ransomed. Or if you're an outside consultant, you get the call from the client going, hey. We got this email that, somebody clicked on, and we think we you know, now we've got this ransom note, and we don't know what to do. From your perspective and based on your experience, kind of describe, you know, having been in the CISO chair, what that first few hours kinda looks like. Yeah. First of all, I'd say it does tend to be a little chaotic whether you want it or not. You can't because, again, you're disrupting normal operations, and this isn't something that happens every day. So first of all, it's just what's really going on. So you're trying to immediately try to you're me as the CISO. I'm gonna be asking my team, what the heck's going on? Is it real? Is it not? All those kinds of questions. So just getting a baseline of reality is the first very first. And then you gotta make the first call as a CISO you've gotta make then is, do I escalate and actually implement our IR plan or a disaster plan or whatever plans you may have, assuming you have a plan even. But to me, that's that first thing is making that first call as to whether you escalate or not. And then when you escalate, what are you gonna actually say? It sounds easy, but when you all you can say is we think we are being attacked. We think something could be. And, again, I think that's the very first days. And then you realize something is happening, then you gotta make that second big decision, which is, do you turn things off? Do you shut things down? And that that is the biggest one, which sometimes you obviously need the CIO group to help make that, maybe even the CEO group. But at times, things may be such an emergency. You gotta make that call yourself and just take that risk. Well, I'm shutting it down. Yeah. I mean, there's definitely a case of, like, is this chicken little? Is the sky falling or is it really like, is this really an incident? And I think that that's, you know, you get some, you know, low level analyst or some junior analyst who sees something and thinks that it's a lot bigger. It you know, there is a lot of, like, hey. Let's first, let's take a breath. Let's, like, figure out, like, what's really going on. Also, understanding just where are we at in the process. You know, when you come in as an outside consultant, especially, things have already happened for a period of time beforehand that you've gotta kinda figure out. But even if you're, you know, the CISO coming in, your team's been investigating this for some period of time beforehand, and it's a lot of it is just like, okay. Where are we actually at in the process? Yeah. You know, you never wanna get the call. I I certainly remember very vividly getting a call from a a client at ten thirty at night on Father's Day one year going, we think we've been ransomed and, you know, asking them, okay. Well, when do you, you know, when did when did you discover this? Oh, we discovered it seventy two hours ago, and the ransom okay. Well, can I see the ransom note? Ransom note says, you know, hey. You've got forty eight hours to pay. So you're at first, you're like, okay, guys. This incident's a little bit different. Let's kinda talk through what you're actually seeing. Let's figure out whether or not this was a real thing or not. And, hey. You know, at this point, we're kinda past the point perhaps that we might have to make certain decisions. I mean, the other aspect of it just is making sure you know you don't alert the wrong people within the organization and raise an alarm bell that you shouldn't. There's obviously a lot of pressure, a lot of complexity with that decision making. Yeah. How important is it for an organization to have a plan ready to go, an incident response plan ready to go, and actually make sure that they have tested it and are ready so that those first few hours, which are automatically chaotic, are less chaotic? To me, there is nothing more important than I than doing the plan and test plan and test. If you have not done that, I and, again, this is just my interpretation of it from my experience. I've seen other companies that are better prepared, and we've all read about all these big incidents that have happened. Some went well, some didn't go well. And it always comes back to how prepared you are to make those critical decisions right off the bat. And if you have a practice, it's awfully hard. Even figuring out who gets to make the decision. If you haven't practiced, you're you're gonna be everyone's gonna be wanting someone else to make the decision, because no one wants to be have their neck on the line for this. So it ends up being to me, that is and I don't care if you're a huge fortune five hundred company or a small company. It's the same thing, just different scale. You need to be prepared because when there's an emergency going on, you don't make decisions the same way. You may have the the person who makes a decision may be different than what it would normal operations would be. So to me, there's just a lot of change that you've got to focus on. Yeah. Absolutely. And I think the other thing that's really important to keep in mind is, you know, as chaotic as it is and as quickly as decisions are being made, there is a value in stopping, taking a breath, and making sure you don't make a decision that seems at the moment to be the right decision, you have to still consider the implications of the decision. Because the one that I think of, and I've had new I had numerous incidents that I was brought into where in the very first few hours, the decision was made. We've gotta get back up and running. And as counterintuitive as it might seem, there would wouldn't you agree that there are times where not getting not or resisting that urge, not getting immediately back up and running actually is beneficial in the long run. And the reason I bring this up is I had numerous examples where we would come in as a forensic examiner, and the logs have been wiped because the decision was let's get back up and running, and they are already restoring from backup. And now all the forensic evidence is gone. Yeah. And don't forget, it trying to get back up if they've already if someone's already invaded in your shop, you're likely gonna get invaded again. Restoring doesn't work until you understand where the damage is coming or where the attacker is attacking so that you can at least stop that. Otherwise, you'll restore, and guess what? There's still gonna be malware sitting right where it was. Oh, yeah. No. That's a that's a really valid point. I remember a couple of incidents that I worked on where it was four, five, six months prior that they had actually gained their access. Yeah. Or or that the backdoor or whatever vulnerability existed. And if you won't you say, oh, I'm gonna restore the last week's backup, you're still not actually getting rid of them because you haven't done the root root cause analysis. And I know that there's a lot a big challenge with, you know, everybody's very sensitive to the times tables and the reporting timelines that you've got. You've got seventy two hours if it's GDPR and all this stuff. There is a real value, and I think what I hear you saying is there is a real value in waiting and actually being deliberate in what you do and not just saying, I gotta hit this deadline because you may reveal information that you then have to walk back, and it's very hard to walk back if you've revealed it. A great example of that. We had a one of the places I was a CISO, large, large company worldwide, but there was a incident where a one of our divisions over in Asia, in this case, was being broken into. And their people decided to immediately go to the press, which had all kinds of problems because judy like you said, it had to be walked back later because, one, we didn't know what was was really going on. Number two is that person wasn't authorized in our company to be a spokesperson to the world for it. So, again, more trouble we got. And I can go on and on, but you that's an occasion where you you've got to follow if you've got a plan, you follow the plan, which, again, makes less mistakes, but you do have to breed and make solid decisions. The the only exception I ever had, we had an incident where we our SOC identified a exfiltration of data, and we had the ability to see where the data was going. And it was going to in this case, I think it was China. And then we it took a little while for our SOC team to realize we have no operations in China. So why would data be going there? And then we went through the the mindset kind of thing of, well, was it just an employee that was sending some matter? Nope. There's PII involved in this data. So that's an occasion where we had in advance. We had talked about that, and we said, if you have the ability to shut it off temporarily, you have to make your own judgment. At times, that's not good either. If you're a healthcare organization and you shut something down, that means maybe you can't deliver healthcare for a while. You can't actually. But, again, it's weighing that difference. And in this case, I had made that decision that if they know PI has been extollurated at a certain rate, cut it off. And Yeah. I mean, it's the old it's the old do I pull the plug or not? If you don't walk up on a computer that's seen some sort of issue, do you pull the plug or not? You know, that was always the debate, you know, in the forensic world. So no. That's that's that's a really valid point. So, you know, kinda jumping to the end and then, you know, kinda working backwards. You know, one of the things that every incident response and really, you should be thinking about almost from the beginning, even though it's the last step in the process. It's the one that it's so much easier to start thinking about it at the beginning of the process than it is to try to go back, and that's when you do your lessons learned at the very end. So it's important to kinda start as you're going through the incident, identifying where those areas where, hey. You know what? We had this incident response plan. We tested it, but we couldn't have been anticipated that this was gonna happen. We could have anticipated that, you know, these this domino effect would occur because it was something just totally random. I mean, you and then nothing ever goes to plan. Yeah. Well and, I mean, you think of some of the, you know, the catastrophic events that have happened in history and, you know, even in the last, you know, number of years, and you it's hard to anticipate all of them. I mean, you know, it's hard to talk about business continuity, incident response, disaster recovery without talking about things like, you know, massive terrorist attacks and, you know, that we're this this is, you know heck the Microsoft event that just happened. Yeah. That to me was one that came out of left field because it was it affected Microsoft, but it really affected a massive number of customers. Oh, yeah. What do you what happens if your if your vendor pushes a piece of software outside of the normal channel where you've done everything you could to prepare for it? Yeah. I mean, that's a great example. The I mean, the other one, it's hard not to address is, you know, we're two weeks out now from the anniversary of nine eleven. I mean, how many people planned in their disaster recovery plans for something like that to happen? It was it was so hard to fathom that it would happen that right. It wasn't part of people's plans. You have to respond to those things, and you have to address them. You have to be prepared to do that. You know, what are some of the things that you've seen what are some of the strategies that you've seen be the most effective when responding to incidents and when thinking about, like, mapping out the lessons learned in while the incident is going on? Well, first of all, on the doing the latter first is the, how do you get the lessons learned? The way I've always done that is someone on the incident response team, as soon as we activate the plan, the incident plan, immediately, I have somebody who is note taking. In other words, every time we get together and make a decision, we have the time, what was done, who made it, the whole thing. And it's not to blame anyone for anything later on. It's so that we learn how we did through the process because it's not generally the technical things that are the instant response problem. It's people. Well, and you also wanna have a real good accounting of what's happened because Yes. Three hours from now, you're gonna forget what you did three hours ago because the stress and the Right. You know, the the all of the adrenaline and everything is pumping. So having that person who's, like, that note taker helps when you go back to write your report even. Yeah. Forget lessons learned, but even the writing the report or reporting out. So yeah. Agree. And then, some of the key strategies to me upfront of what makes for success is this to and again, I'll and I'll I'm gonna say this ten times, I'm sure, before we're done is plan, practice, practice, practice, practice. You if you have not practiced, I guarantee you it's gonna be much worse. It's Oh, yeah. No different than I took up pickleball probably about six months ago. Horrible. Never practiced. Guess what? You practice a bit and you get better at it even if you can't anticipate all that's going on. It's the same thing here. You have to practice. So Oh, yeah. I mean, I have a I have a twelve year old son, and he's every month, he'll say to me, we just had a fire drill at school, and he's like, they they they stopped us from doing whatever work we were doing. I'm like, yeah. Because they want you to practice so that when something happens, you know what to do, and you don't have to even think about it. Right. So, I mean, we can go somewhere else. Yeah. It it sounds so simple, but very few companies on my mind, and there are plenty that do, but that actually do that real testing. And it's great to do a tabletop and to talk about it, but you gotta make it as realistic as possible because you are gonna be under a lot more stress when it really happens, and you need to make sure it actually works. And to have a written plan in advance and just things as simple as how do you get the team together for who gets to get who is the team when there's a real we'll call it a security incident in this case, but a lot of times you also need someone from your comms group. You need legal involved. You need senior exec representation, maybe the CFO, maybe the CEO. All that should have been decided wrong in advance so that you can Yeah. And you definitely want marketing involved. I mean, we've seen numerous instant instances of poor marketing, poor communication after an incident. You know, I there there was one with one of the big credit bureaus a number of years ago where they just fumbled the, you know, the the outreach and the public, perception of things. They were Yeah. We don't really know what's going on, and it's like, okay. Well, what then why are you, you know, why are you telling us, you know, nothing, you know, or telling us stuff and then having to walk it back, you know. It's all about reputation in that moment. Right. You know? And I think the other thing that you that you brought up that's really important, you know, we just concluded. And, earlier this month, we talked about our business continuity and incident response in the cloud working group that we had. And one of the things that kept coming up was this idea that when you test, the real realistic nature of the test is so important, and there's a proclivity or there's a tendency for organizations to want to schedule their testing when everyone can be in the room. And that's not necessarily the most optimal situation because when an incident happens, your CEO may be on vacation. You know? Your CEO may be out of the country. Like, you have to know what you're gonna do if not everyone is where they need to be when the incident occurs. Yeah. I'll give you another great example of what you probably don't wanna do, and this is a place where I was. We did a semiannual test, and it was a good test. I mean, it did a lot. The only problem came is we were and we physically we went from one data center to another data center, fairly common in the old days when there was not a lot of cloud and all of that. The problem came, though, that it is we transferred and brought up the system there. Something went wrong, so we called a time out. In other words, we'd stop the clock of the incident and just they went away for an hour and fixed it, then we started it up again. Interesting approach, but it's not realistic. You don't you can't call a time out in the middle of a real incident. That's the whole purpose of it. So it just amazed me that we did that. And we we tried over time to get that changed, but it's hard. Yeah. Yeah. Mister Ransomware Group, please, like, give us an hour. Like like, come back. Let's let's start this again in an hour. Sorry. Yeah. We weren't ready. We had we had a big issue. Like, give us just a few minutes. Let us change our clothes. We'll be back. You know? Yeah. Like, it doesn't happen. But I think your the the point of of all of that is really important, which is you've gotta you know, it's almost you've gotta expect the unexpected. You know? You've gotta be prepared that things are not gonna go the way they want. You know? I always incident response, I know it's been used multiple times, but I always go back to the Mike Tyson quote. You know, everyone's got a plan till they get punched in the face. Like, that is incident response. So, you know, I mean, it sounds cliche, but it really is. There is a huge amount of truth to that. It's, and, again, it you have to accept that it's never gonna go. And that's the hardest part is to make sure you have a team, the incident team, that may not be making every decision, but they're orchestrating it. So they collect all the data throughout so you don't make duplicate decisions or decisions. One makes it one way, one makes it another. That act I've that's happened to me too. One said, unplug it. The other said, don't unplug it. Plug it back in. And we had a fight going on amongst ourselves instead of actually solving the problem. Well and how important and I'm gonna ask the question, then I want you to kinda talk a little expand it a little bit because, you know, into what are the other kind of characteristics that are important. But, you know, how important is it for the CISO or the incident commander, whoever it might be, if it's not the CISO, to to really be the one who kind of calms everybody a bit because you would talk about the chaotic nature. Everyone's gonna be running around, you know, here on fire. Somebody's gotta slow everybody kinda there is value in slowing everybody down. That's, you know, to me, seems to always be the most import one of the most important is that the CISO has to say to everybody, like, I understand how important this is, how serious it is, but let's slow down a little bit. Let's take our time. Let's not freak out. What are the is that an import you know, has that been your experience? And then what are the other kind of characteristics that you think are and skills that are the most important for the CISO to exude Yeah. In that moment? Yeah. Me personally, I probably I don't make a great incident commander because I get too excited. And I'm gonna be honest I get into it. So in my case, incident commander was my chief of staff. In this case, she was she just was calm. She took all of the data in. It didn't matter if it was a CEO talking. She could she had that ability to still say, can we wait just a second? I mean, always polite, but she was in control to follow the process that we had agreed on in the past. So, to me, that is a massively important role. And it's I think too often, it CISO will do it or this person will do it. It doesn't actually need it's a skill set that a lot of people don't have, including myself. Oh, yeah. And being self aware enough to say, listen. I'm not the right person because I'm gonna go chasing down rabbit holes. I need that other person who's going to keep us calm, who's going to, like, make sure we're following the process because our natural humanity is to just to jump in and start solving the problem, and that may not always be the most effective thing. Yeah. I think that even it's if you think about it, if as the CISO, your head's gonna be if it's a cyber incident, your head's gonna be into all the pieces of your your group and all the security, the monitoring, and figuring out reality of what's there. Now how at the same time can you then communicate with legal, with comms, with marketing, with this there's it's just not realistic in my mind, especially for a larger company, that the CISO could be the only one sort of emceeing this thing. Well and how much does some of the, like, changing landscape and CISO liability over the last couple years also kind of drive you towards the CISO probably shouldn't be the one controlling the situation because they've got, you know, they've gotta worry about their head on a platter even. Oh, and that's that's true too. Don't forget. I mean, that there's business damage done, then it's not really an incident. Right. And how many CISOs are that involved with every aspect of the business to understand how it works? You gotta have the business folks involved in this more. They'll ultimately make a decision whether offline, online. My job is to explain to them what's going on and to make sure they understand that the risks of doing it, of not doing it. That's my role in my opinion. Yeah. So, you know, we've obviously talked about the biggest pitfall being not testing and not having not sticking to your plan, not having you know, not taking that minute to just take a breath and be calm. What are some of the other big pitfalls that you see that you've seen in incident response? I would say, in advance, not just practicing that, but making sure you've communicated with the entire company on if there's an incident, you don't respond outside of our the walls of our world. In other words, we have a designated person in this company that will be the spokesperson for this company and not you. And I've been through two incidents in my career where that did not happen. In both times, we had to, as you said, walk back and try to explain away where you lose a lot of credibility. So number one, I would definitely put that as another critical upfront thing in along with planning and all of that. The second is from a CISO perspective, if there is ongoing damage being done, in other words, let's say data exfiltration is happening then, I think you do have to make a quick decision then on that one. And at times, even though you're gonna the risk and liability that we just talked about for CISOs, at times, you gotta be willing to still make that call. You're a there's a it's a C-level role for a reason. Right. You got to make decisions, and I've talked to a lot of CISOs that basically said I would never make that decision. And at times, you got to and live with be willing to live with the consequences. And, but, again, taking that a step further is in today's world, if you don't have director's liability insurance coverage, you really shouldn't take the job. Yeah. No. Insurance coverage, you really shouldn't take the job. Yeah. No. Absolutely. And and and, you know, I think from my experience, I mean, some of the going back to kinda what I was saying, I think the bit one of the biggest pitfalls I've certainly seen from the technical side is that the tendency or the desire to immediately get back up and running. And Yeah. You know, as a forensic examiner is somebody who's gonna come in and do, because usually, most organizations, even if they have an internal security team, they're gonna call in an outside security team to come in and do the investigation because Yeah. Of independence and objectivity and just Hopefully, I got them on retainer already. And hopefully, you've got them on retainer. Yeah. But there are lots of reasons to bring them in that don't necessarily mean that you've done anything wrong as a team. It just it's extra bodies. It allows your internal team to focus on the technical things. But if you have taken decision you know, going back to the example, if you've made the decision to get back up and running and just start restoring from backups, you may not have realized that you destroyed forensic data that was important to answer the question that you're going to get asked, which is what happened? How did it happen? How do we know that it's not gonna continue to happen? And you're trying to thing is the other thing is your internal team that one of the biggest pitfalls is that the internal team will kind of act as it act on the premise that they just they know everything. They're the experts in the system, which, yes, they are, but that can also cause some blinders to go on. And you might Oh, yeah. Say, oh, yeah. This is not that big a deal. We've seen this before. Whereas an outside perspective might go, hey, guys. That's actually a really big deal, and you need to be concerned. Yeah. Another advantage of the outsider coming in, whether you like to hear this or not, is an outside person typically gets listened to a little bit more quickly than an internal person. It's just human nature to a degree, but the consultant said it, so it must be true. So I've also found that getting the support of that outside person to make quick decisions or quicker decisions Yeah. Pays off. Well, and and back to the point of just keeping the calm and outside perspective is and outside individuals gonna help you. Like, they're not gonna get caught up in as much of the emotion of it because they're coming in to be an advisor to help guide you, and they've seen this hundreds of times. I mean, the number of times that I would come into an organization, and this is the first time they'd ever had an incident. And they're operating based on what they, you know, what they've kind of practiced and planned, but I'm able to come in and go, well, what you're seeing here is actually something I saw three weeks ago at another company. Let me tell you let me help you kinda not make the same mistakes they made can be really valuable. So Absolutely. Don't be afraid to use outside help. It's not a sign of weakness in this case. I'd agree. And it's I think also the, most companies, unless they're incredibly large, don't typically have a deep forensics team. Yeah. And it's just being realistic. And bringing in that outsider, like you said, has done it a hundred times, they're gonna help you make better decisions, quicker decisions, consider aspects that you haven't even thought about still because it's possible they've actually seen the exact thing that's happening to you, whereas you may have never seen it. So, yeah, I agree with you. Yeah. And I mean and and the other thing just to keep in mind, you know, just the the nature of digital forensics and forensics in general is your job is to sit around and wait for something to go wrong. Most organizations have no real desire to have somebody on staff whose only job is to sit around waiting for something to go wrong. So if you have the forensic skills and you come into an organization, they're gonna have you doing all kinds of other things that are gonna pull you away from being able to do the forensics when the incident occurs. And, you know, so that's why you typically bring in the outside consultants because they're able to have, you know, not just your organization, but a bunch of other organizations that are all having incidents when you're not so that when you have your incident, they can come in. So it's there is benefit to it. So I'd be remiss if we didn't talk a little bit about the role of technology and some of the advancements we've seen over the last couple years in, technology and how those can maybe aid incident response. Yeah. You know, we've we we we've all we all know, you know, or most security professionals are familiar with the idea of SIMS, you know, security event management and bringing everything together. Yeah. We've talked a lot. People know a lot about the soar and, you know, some of the automations that can you can put in place. So instead of having to look at every single phishing email because for anyone who's worked in a SOC like myself Yep. Reviewing phishing emails gets very tedious, very tiring, and very just mundane after a while. And you're like, this sucks because I'm seeing the same. It's literally just checking a link and going, yep. This is bad. This is bad. Like, the number of emails that get reported as potential phishing that are, in fact, phishing is almost like, I think, like, eighty five or ninety percent, I feel like. Yeah. We don't even report the volume of that to the board anymore. It's so I mean, it Yeah. So You know? But where can have you seen places where AI and some of the more, you know, advanced automations, some of the decision making tools can can really be beneficial. Yeah. I and I'm not probably not gonna go to the word AI yet, but it's, I do think there's machine learning. There's all kinds of tools that have advanced a lot that are using, I'll call it, computer smarts, whether it's truly AI or what it may be. But to me, the power of that is it can help you get to what damage is done quicker. That is the by having and it again, it only works if you've done this in advance. In other words, you have to have already accumulated all your security data, a log data from all the sources, figured out what report, how to report that up where it's meaningful as far as is something is there really a spike that's happening? Where is it happening? Do you have the ability to then dig down quickly and find out where it comes from? All those questions, to me, that is that there is certainly technology there that I think that is incredibly helpful. The other side to me is, and, again, this will be an interesting one. Most incident response plans, the overall the large plans, I'd say the majority of those are still binders that have been printed out because you can't count on your network being up to even get to that data. So I think that another aspect, and this is one that our company is even coming out with soon, and I'm sure others are working on this too, is it is a creating a cloud space, if you will, where your instant response, all the dozens and dozens of components of it have a place to rest where you actually can get to securely outside of your network. Yeah. And you bring up an you just raised a really interesting point that I wanna pull on a little bit, this idea of having out of band communications and technologies and things in place. Talk a little bit explain for anybody who might be listening, who may not understand kinda what we're talking about here Okay. The importance of this concept of having an out of band communication Yeah. But also even just out of band like, having your binder on, you know, a OneDrive that is not connected to the rest of the network. Correct. That's to me, it is far more logical to think that that's going to happen than not. If your network is down, my guess is today, most companies, if you're gonna communicate, it's through Teams or some kind of an you know, it could be email. It could be phone like we're doing. We're recording this right now. Assume none of that's available. Your entire network is down. So how are you gonna call someone? You even have their phone number. How are you gonna reach them? Oh, and, again, there's you can go satellite. You can go out of band like you talk, but that's something you actually have to think about. I know at the healthcare organization, I was a CISO app. We took the top ten people the company, although some of those overlap, but we took the top people that would be involved in any incident initially, and they all have satellite phones. Once a quarter, they all we all had to get on a bridge line with that satellite phones to prove that we could make it work. If you don't ever use it, it's sort of hard to use. So but, anyway, that's just one example. But I think having you have to assume your network is not available. And, again, the space that I work in today in Active Directory, we always tell everyone, if Active Directory is down, virtually everything you use is Yeah. Down. You can't log on to anything. So you have to assume that's possible to happen. And if your system is ransomed, you're not gonna be able to send text messages on any of the applications or access anything. So that idea yeah. And I think critical infrastructure operators are very familiar with this. There's Yeah. All kinds of technologies and opportunity and and things that I know Department of Homeland Security gives them. You know? Yes. W gets and all of those types of things that are It's you know, those those out of band technologies. And, you know, for those of us who grew up in places where you know, I grew up in Florida. I live in in Houston. For those of us who are familiar with hurricanes and other natural disasters, the incident responders here who are responding to, you know, live natural disaster incidents have out of band communications. Cyber is no different. And No. And it nothing else. It's more important because the options and and some of the kind of planning is less, deliberate on that task. So that's a really important point. So well, even think about when the how heavy the airlines got hit just with that. And it was a hack. It was just a bad software cut to them. Right. But think of the impact it had. I have no clue. I mean, obviously, they couldn't schedule airlines, and the airlines shut down for a period of time virtually. But at the same time, could they even communicate well? I don't know the answer to that. If you're, at least, a lot of companies I deal with are fairly dependent on Microsoft. And if Microsoft's not there, then what do you do? Yeah. Absolutely. And, you know, as as as we're starting to get kinda closer to the end of this, you know, I wanna I wanna kind of before we go into some of the questions, you know, that we've got, Yeah. From the audience, really, I want to address is if you had to give a team, you know, kind of three, four, five practical strategies, things to keep in mind that should be on your, like, top five list of things when an incident occurs. What are kind of those those top five key most important things? Okay. Well, I'm as I said, I was gonna say this ten times up front. Plan, test, plan, test. So we'll take that as one of those just because you have to include it. I count that as two, actually. I think you said it twice, so we're good there. So you Okay. You got But if you and if you don't say that at first, then I'm almost wasting your time with anything else that's out there. I think that's the second. The second is you need and, again, it's amazing how often you get into arguments over what you're going to restore in what order. You have to accept that when there's a major outage caused by security incident, you can't recover everything simultaneously. In other words, you have to, in advance, have decided what app needs to come first. And guess what? Security doesn't make that decision. Others make that decision. So if to me, that's another that would be point three in my mind then is in advance, you've gotta have a plan on what's gonna get restored in what order. Does that make sense? That makes complete sense. And, you know, again, it goes back to a a conversation we had earlier in the month about the importance of having a business impact analysis and knowing the order of things. You know, I always I use it over and over and over as an example because it's just to me, it's the most vivid example of this idea that you have to have prioritization is the scene from Apollo 13 where, you know, Jason is just sitting there trying to figure out how do I get to twelve how do I not go over twelve amps? You know? You have to prioritize the order in which you turn things on. I think it's the same thing. And, you know, especially, you know, again, one of the things we talked about early in this month in this month was this idea that you've gotta know, you know, if you're in the cloud, it's knowing knowing what your agreements with your vendors are, but and knowing where you sit in the prioritization of that vendor. Because going back to the big IT outage, if you're if they've got a hundred thousand customers that are that are impacted and your customer nine hundred and ninety nine you know? You're gonna be waiting a while. You're gonna be waiting a while, and you could you know, they're gonna say, well, if you just paid this much, you know, if you just paid the extra forty dollars on Southwest, you could have been, you know, in the top Yeah. You know, one through fifteen, a one through fifteen, and you go, oh, well, if I had known, maybe I would have done that. It's a And you have to plan that that's going to happen. Oh, yeah. Everyone can't be in the top ten of a a vendor's list. Oh, of course. Of course. If you're not there, then you can say, well, you make the assumption that these guys are gonna take a a while to get to me. So what am I gonna do to minimize the damage while that's happening? Yeah. And I think from my perspective, the practical strategies, obviously, the first, you know, as I've kinda said over and over is remain calm. Like, it is a very hard thing to do. It sounds easy enough to say it, but I as as somebody who has been in incidents before as you have, it is a lot harder when you're in that incident and the adrenaline is pumping to really stop and just remain calm about it and understand that there's gonna be you're gonna have a better outcome than the calmer you are. I think the other thing that is, for me, is so important about incident response is remembering where you know, think about the things that are gonna come three or four moves down. You know? I played chess a lot as a kid, and I still do. And I almost think of incident response. And anything we do in in security, but especially incident response has to be thinking that chess move three moves down. What am I gonna do? Because when an incident occurs and, you know, you're gonna have to be prepared for that, and you might make a decision at step one, like going restoring everything and forget to, you know, preserve the forensic data. Absolutely. So, you know, at this point, first of all, if anybody's if anyone who's listening, if you've got questions, please feel free to submit them. Please feel free to put them in the question and answer box within the platform, and we'll do our best to get to them. You know, one of the questions that we that we got in, you know, what do you think is the most overlooked step during the first hours of incident response? I'm gonna make it two things. I just can't do one. One, communication. Making that decision, who gets informed at what pace. The second is, I think, in that first hour, understanding that it's not just about if you have an application is down, you need to understand that it may not be the application that's the problem. It could be the infrastructure below it. So you actually need to go down the stack a bit to make sure that you're considering all of that. And that we talked earlier about Active Directory. If it's down, your app's down. So you things like that, I think you have to keep in mind in the early stages. But to make communication and then, obviously, the next step in there is you've got to get your team focused on trying to figure out root cause and what damage is actually occurring. Yeah. And I think, you know, you bring up you you've brought up multiple times this idea of communication. And I think one of, for me, one of the most overlooked, to take that kind of even a step further is making sure to control to the best of your ability or to address the internal gossip rumor mill that goes on in the organization. Because if yes. It's important to make sure that only one person is speaking outward to the organize to to the the industry and to the world, but that rumor mill internally can really cause a lot of trouble because you have somebody who hears something from somebody and they share with one of their friends outside and let Yeah. You know, there's all that natural, like, oh my god. Is the company gonna, you know, are we gonna are are we all gonna gonna lose our jobs because of this? Or you know what? And that What is Inevitably, that leaks too. It will leak to the outside. Oh, yeah. Everyone starts rumors around internally. So that's another key part of that early incident response plan is getting a communication out internally if it's possible. And I think, you know, taking the like, making sure everybody knows their roles and responsibilities even a step further is establish now who that incident commander is going to be. Like, know who that person is and make sure they know that that's gonna be their role because you need that person to step in almost before you step in Yeah. To start managing the incident because you're gonna you know, as we've said, we're you know, we both are are of the nature to immediately start jumping in and forget to, oh, hey. Wait. I need to pull the incident commander in. The minute the incident commander sees you getting jump jumping in, they need to know to activate and maybe even activate without being told to activate. So Yeah. That is commander's critical role, and it's technically not a deep technical role. Oh, no. It doesn't have to be. It just has to be somebody who's organized, who can remain calm, who has grace under pressure, and, you know, can help organize everybody, has the relationships where people will stop and listen to that person. You know? Yep. It needs to be somebody who can play the hall monitor that's Right. You aren't gonna just try to run past. And I'll bet you you've seen a fair number of incident commanders who do not fit that bill. I have been the incident commander who doesn't fit that bill. I'll be the first to admit. I've been the incident commander who goes who who looking back, I'm like, I should never have been the incident commander on this because I was not the right person to take us. Yeah. Person's got no limitations. And part of it is just, honestly, I'm a a naturally curious person, and I like the investigative side coming from forensics Yeah. Side. So for me, it's jump in, start figuring out what's going on, try to stop the attacker from you know, it's always about there is a a bit of bravado amongst forensic examiners of wanting to be smarter than the attacker. So Yeah. Hey. I caught you, and I was smarter than you are. So I'm not the best incident commander, and I'll be the first to admit that. Gotcha. So, you know, I think and I think we've addressed this one, but I definitely wanna wanna kinda double click on it is, you know, how do you balance speed and thoroughness when responding to an active cyber incident? And I think this is important because we all hear, hey. If it's you know, if you've got European data, you've got seventy two hours to to notify, which if you're if it's a supply chain thing, you might not have all the full seventy two hours. If you've got customers that have to notify their customers, you might not have seventy two hours. How do you but at the same time, we've said you need to know what's the root cause. You need to be ready to answer all those questions. How do you do those two things at the same time? Again, it's all balance is what it's coming down to. And so much of that depends. You in advance, you need to understand the impact of being out. If you've got seventy two hours, that's one thing. What if you're a trading floor and billions of dollars are going through that every Right. Second or two? That changes your potential of when you may have to make a quicker decision. And, hopefully, in advance, you've talked that about with people and talked about if this is occurring, this is what we will do. You can never plan every scenario, but to me, you've got to balance the two because the worst you can do just as much damage by reacting too quickly as you can by not reacting quickly enough. And it's that balance, and imbalance comes from knowledge of the operations of your business. You have to know your business. And I think there's also value in determining you know, we've talked about a lot about the decisions that you need to make in the immediate, you know, aftermath of discovering an incident. There's also value in understanding which in which decisions decisions you don't need to make right away. Like Yeah. I mean, there there's certainly been a lot of talk. I've heard a lot of talk within our community about, you know, materiality. And, hey, you know what? You may not necessarily need to make materiality decisions while you're investigating the incident. Finish investigating the incident, then, you know so having those kinda but I use that as an example. That may not may or may not be a decision you can defer. But if there are decisions you can defer, knowing those what those are ahead of time is very beneficial. Well, and the quicker you can get on a line or a phone with the right people, the right senior management, or the right people to help make that decision, you're gonna make a smarter one. It shouldn't be a decision typically that you make on your own as a CISO. And and just as important as having the forensics or marketing on retainer is having outside counsel on retainer of people who have done this hundreds and hundreds of times. So we have about five minutes left. I, you know, if I know there have been some questions we haven't had a chance to get to. We will certainly, follow-up with people on this, and we'll be talking more about this in other ways. Kind of final thoughts, anything that, you know, kinda stay sticks out in your mind, and then, you know, tell us a little bit about Semperis. Where can we find you guys? You know, kinda what explain it for people who may not Okay. Know who you guys are. Sure. First of all, I the last thought I would say is and I'm gonna go back to the last topic we just talked about, which is the legal counsel. That becomes one of the the most important communications in any security incident quickly because there's so many regulatory, legal, contractual. There's so many things that have an impact that you may not know about, which they do. So I think and also how you communicate. All of those things, I think, is something that you don't may not want to involve legal upfront with all that, but it's one I think you have to. Absolutely. And then and then just let us know, you know, for anybody who who real quick, you know, what is Semperis? Where can we find you? Semperis, we, deal in identity. Protecting your identity systems is the best way to do it, to describe it. Typically, I mean, most people's identities are stored in Active Directory, in Entra, maybe Okta, a couple others, but we focus very narrowly on protecting those systems. And it's for the very reason if you take virtually every one of the last ten, twenty major breaches that have occurred, Active Directory has been right at the core of it, especially ransomware. You shut ransomware if you shut AD down, none of the apps work, whether I don't care what industry you're in. So we focus on three things, hardening that system up before an attack occurs, continuous monitoring of that system of Active Directory or Entra with the ability to actually do repair. If you've noticed something, someone adds a new group admin, you can immediately undo that. You can have rules up front. And the last thing that we do then is if the bad happens and you do get ransomware, we help you recover very quickly. And by very quickly, typically, we cut about ninety percent of the time out. And we've got customers that historically, when they got hit three to four weeks to recover Active Directory, now it's measured in a few hours, and it's actually tested on a quarterly basis typically. Great. Well, thank you so much for being here, Jim. Thank you for this conversation. Thank you all for attending and and participating. Please, you know, stay in in touch with our website, stay in touch with our our social media. We are gonna have a lot of these conversations in the, coming weeks. We're gonna continue talking about this topic because it's certainly an interesting and important topic. Again, Jim, thank you for being here. Thank you, Semperis, for sponsoring this for us, and, hope everyone has a really great rest of your day and in an upcoming weekend. Thank you very much. Thank you.