In January 2022, FlexBooker discovered they had a problem. A hacking group known as Uawrongteam had penetrated their AWS environment due to a misconfiguration. It was the second successful attack in as many months, stealing personal information from millions of FlexBooker users, including driver’s license numbers, IDs, and passwords. The hackers planted malware that granted them unrestricted access to FlexBooker systems, including the ability to change user appointments for things like health care, such as COVID-19 vaccination times and locations. The stolen data (~19 million unencrypted HTML files) was soon put up for sale on the dark web, creating identity theft risk for at least 3.7 million users.
The attack resulted in FlexBooker losing numerous patrons, causing significant reputational damage and revenue loss. Amazon had to step in to resolve the attack, a process taking nearly 12 hours, because FlexBooker was unable to handle it on its own.
All companies today face risk exposure to cyber-attack. Small companies get hit as well as large, public, and private companies. But the effects of cyber-attacks do not need to be as bad as what happened to FlexBooker; damage can be avoided, reduced, or mitigated if you plan in advance and take precautionary cybersecurity measures.
Incident response readiness should be a core component of every IT cybersecurity program. A wise leader will have a team already in place, trained on how to react to incidents, practiced in responding to unique situations under time pressure and at moments of high stress, and well-versed in the organization’s priorities if system or data triage becomes necessary. In our work at CREO, I train teams at many companies on how to think during a cybersecurity disaster, and I develop custom-tailored scenarios for them to exercise their skills responding to an incident that is realistic in its scope and applicable to the organization. Here is a subset of what CREO teaches these teams:
- Have a Defined Process: An Incident Response Plan is critical, because in the moment when an attack is occurring, the last thing you want your team doing is trying to figure out how to coordinate and drive a response. Minutes are precious. They don’t have time to think; they need to know what to do without thinking about it. Having a written plan (stored as a hard-copy offline) enables immediate action. Practicing the process at least annually ensures that the team is readily able to implement that process, so you have no delays between incident detection and the response team diving in.
- Get The Right Team: Many organizations plan for IT to manage information security incidents. This makes sense; after all, cyber-attacks hit systems, and IT manages those systems. But limiting the response team to just technology personnel ignores the numerous other impacts that incidents may have on an organization. What if the attack results from a malicious or incompetent insider? HR will have decisions to make. What if the attack breaks key operational processes? You need an operations expert to give the rest of the workforce guidance on what to do while the problem is being fixed. Do you hire a professional forensics team to try to catch the bad actors? You’ll need legal counsel, and someone with budget authority to make this kind of decision. Most important of all is to have a communications plan; what do you tell your clients, your employees, or regulatory authorities about the loss of private data? How will you shape the narrative if the event is likely to become publicly known? This skillset is closer to public relations than to IT. A good security program will have all the right resources pre-identified, well trained, and on speed dial, so that every dimension of the incident is met with action, not just the technological impacts.
- Know What Is Important: Incidents should be judged by their impact. A system used for internal employee satisfaction surveys is less important than a system that runs a sensitive proprietary manufacturing device. The CEO’s laptop is more sensitive than that of the average worker. If the attack cripples every system, then everything will be broken at once, and the team will need to know what to fix first. That means that part of incident readiness is planning incident triage and system triage in detail. How bad does it need to be before you activate the incident response team? If the incident is really bad, how long can each system be down before it hurts the company, and which ones are most important to bring back first? Assessing system criticality and data sensitivity in advance allows the team to quickly prioritize between multiple objectives instead of being paralyzed by numerous conflicting priorities.
- Train and Test: Responding to incidents requires a very different skillset than the skills the team members use every day. It requires making very fast decisions in the face of very high ambiguity, with high stakes, and correspondingly high stress. In situations like this, people need structure and experience to fall back on. By repeatedly training on the process, and then testing it using tabletop or live red-team exercises, the team can build muscle memory that can make the difference between ending an attack in minutes vs. hours, with megabytes of lost data vs. terabytes. Team members will ultimately become accustomed to operating through stress and ambiguity, to acting decisively based on limited knowledge, and to quickly gathering the knowledge they need.
When we create cybersecurity test scenarios for client training at CREO, we intentionally give participants very little information to start with and make them talk us through the actions they might take to gather more intelligence. We present them with difficult, sometimes no-win decisions, so they get experience making a call and moving forward, and experience dealing with the fact that sometimes there are no good options. We simulate the added pressure of CEOs breathing down their necks with frustration, media outlets calling them for quotes after being tipped off about a breach. The goal is to make the team a little bit uncomfortable, which serves a twofold purpose: first it prepares them for the stress they might feel in a real event. Second, it makes them very eager to avoid getting hacked in the first place.
At the end of each session we debrief, and discuss the experience, looking for lessons learned, things they can do to be better prepared, vulnerabilities they noticed during the exercise. We want teams to learn from their training and testing experience, so they can avoid the FlexBooker situation, with two hacks in two months against similar vulnerabilities. Fool me once, as they say.
As a trainer, my favorite part is when I see how teams come together during these after-action debriefs. Often, they are inspired by the experience, suddenly aware of the realities of their security posture, and internally aligned with drive and focus to make real improvements.
Trained Incident Response team members take on a new perspective toward information security risk, they gain confidence that they know how to prevent attacks from happening, and handle those that do happen. In short, they become prepared to respond quickly and decisively to reduce business disruption and reputational issues.
