4 min read
AI speeds up attacks, so preparation must keep up
On a Thursday at 10:40 p.m., while the IT department of a 300-person distribution company is fighting ransomware, the finance director receives a voice message from the CEO, who is traveling. He asks her to pay, that very night, a supplier whose bank details have changed, and not to go through the usual approval, for lack of time. The voice is his, and the email that follows is written without a single mistake and cites the right order number. Everything is false, built from public recordings and information found online. The company is fictional. The scenario, however, is exactly the kind of thing generative artificial intelligence has made easier, and it forces the question of what it changes for cyber incident readiness.
What changes on the attacker’s side
Generative artificial intelligence did not invent cybercrime. It made some of its steps cheaper and faster, starting with the most visible one, the phishing message. A few years ago, a fraudulent email often gave itself away through approximate spelling or a foreign turn of phrase. Today a message can be written in the recipient’s language, fitted to their job from public information, and sent at scale with variants. Cloned voices and faked videos, for their part, are used to impersonate an executive, and payment frauds obtained this way have been reported. Reconnaissance or programming tools, long reserved for experienced attackers, are becoming accessible to less advanced profiles.
The practical consequence is uncomfortable. The warning signs taught in awareness training, such as language mistakes, tend to disappear, and an employee who learned to look for them may feel safe facing a message that contains none. Awareness must therefore change its object: less spotting flaws, more verifying through a second channel anything that asks for money, access or an exception.
What changes on the defender’s side, and its limits
The defense has the same tools. Artificial intelligence helps sort alerts, summarize logs, draft a first version of a statement or translate a message for a foreign customer, and it saves time where time is most lacking. It would be absurd to do without it. It has a weakness, though, that matters a great deal in a crisis: it errs with confidence, and a wrong answer, well phrased, reads exactly like a right one.
During an incident, the scarce resource is the established fact: what do we really know, since when, with what degree of certainty? A statement drafted by a model from incomplete information will be fluent and may be inaccurate, and nobody has time to check it sentence by sentence. It therefore always needs someone who knows what is true, which makes human review, more than drafting, the real bottleneck. A team that has understood this uses the tool to save time on form, never to decide on substance.
The forgotten risk: AI inside the crisis cell
There is a risk that companies rarely anticipate, because it comes not from an attacker but from the team itself. Under pressure, someone copies the incident log, a customer list or the ransom message into a public assistant to go faster. Personal and confidential data then leave the organization, in the middle of an incident, when it is least able to notice. The gesture is understandable: the tool is open in another tab, the task is urgent, and nobody said it was forbidden. It can nonetheless add a second problem to the first, notably with regard to data protection.
This is decided before the crisis, because it cannot be settled in the heat of the moment. It means saying which AI tools are allowed during an incident, which information is never pasted into them, and who in the crisis cell reminds the team when fatigue sets in. A three-line rule, known to everyone and rehearsed, is enough.
Train for deception, and prepare faster
Back to the finance director at 10:40 p.m. Knowing that this kind of fraud exists is not enough for her: what matters is whether the verification procedure, calling the CEO back on a known number, requiring a second approval for any change of bank details, holds when everyone is tired and the incident occupies all the attention. That can be tested, provided it is put in an exercise. A good scenario now contains an impersonation: a call “from the CEO” asking to bypass the usual approval, a fake supplier email arriving in the middle of the incident with new bank details. You then watch what the group does, not what it says it would do.
Artificial intelligence can also serve preparation rather than attack, by reducing its cost. Adapting a scenario to your sector, your systems and your roles used to take days of manual work, which explains why few organizations practice often. UnderCrisis uses it to generate the exercise from your organization’s context, and the decisions remain those of your teams. If attacks are gaining speed, preparation can no longer remain an annual project: it must be able to repeat as often as situations change.