The state of AI security
AI-powered attacks command attention, but this year’s research points to a more immediate challenge inside organisations. Unapproved tools, employee behaviour and AI embedded in everyday workflows are testing whether governance can keep pace with rapid adoption.
AI threats are an inside job
This report made waves last year by being one of the first to provide evidence of the risks associated with AI in the enterprise. As adoption has grown, so have those concerns. Yet we can observe a tension between where many respondents believe the threat lies, and the reality on the ground.
More respondents (39%) are concerned about the emerging threat of AI phishing over the coming year than any other factor. It’s a valid point. The UK National Cyber Security Centre (NCSC) said as far back as May 2025 that AI would help threat actors to enhance social engineering and thereby “increase the volume and impact of cyber intrusions” over the succeeding two years.
Large language models (LLMs) can do much of the heavy lifting: for bad actors: scraping publicly available data to compile target profiles/lists, crafting personalised messages in perfect natural language, and even throwing in deepfake-generated content to make attacks more successful. All of this can be done at tremendous scale, with capabilities making their way into the hands of more scammers through phishing-as-a-service models with baked-in AI. One report claims AI tools helped to double the volume of phishing emails recorded in a year.
However, threats like AI-powered phishing and vulnerability research and exploit development (VRED) are only one part of the story.
More respondents (39%) are concerned about the emerging threat of AI phishing over the coming year than any other factor. One report claims AI tools helped to double the volume of phishing emails recorded in a year.
What, if anything, are your biggest emerging threat concerns for the next 12 months?
Looking inwards
In terms of actual incidents, the more likely scenario today is of internal AI use creating security and compliance problems, as opposed to external attacks, our research finds.
A quarter (25%) of respondents blame unapproved AI use for their most significant incident over the past year – whether that’s leaking confidential information via an AI tool without approval, or giving an AI tool access to a business/app system without approval. That’s a larger share than those who cite phishing/social engineering as their most significant incident.
This suggests AI is becoming an established source of corporate cyber risk, rather than being viewed simply as a future threat.
Shadow AI/IT (45%) is also flagged by respondents as the number one challenge they’re facing today; whether that’s unsanctioned tools, or IT/AI decisions being taken without involving the security team. A greater share of respondents (29%) claim internal misuse of generative AI (GenAI) is their biggest emerging concern of the coming 12 months than do ransomware (22%). A fifth (22%) of respondents say staff have built AI into important workflows without the security team knowing. The risk is likely compounded by the fact that the same share (22%) of staff have shared confidential or sensitive data with public generative AI tools.
What, if anything, are the challenges you are currently facing in information security? (Top responses)
Shadow AI and shadow IT — unsanctioned tools, or IT/AI decisions taken without involving the security team
Information security skills gap
Securing emerging technologies such as AI, ML and blockchain
Rise of “as-a-service” cyber threats (e.g. Ransomware-as-a-Service, Phishing-as-a-Service)
Securing cloud-based services and applications
The risks associated with shadow AI should be well understood by now. If IT and security teams don’t know which tools employees are using, they can’t protect leakage via DLP. They can’t vet potentially insecure LLMs which may contain exploitable bugs. And they can’t enforce strong identity and access control policies. The organisation could be exposed to potentially significant security, privacy and compliance risks, as well as threats to reputation stemming from hallucinations. The risk is so great that the Solicitor’s Regulation Authority (SRA) was recently forced to issue a detailed warning to the UK’s legal sector concerning AI misuse.
A quarter (25%) of respondents blame unapproved AI use for their most significant incident over the past year – that’s a larger share than those who cite phishing/social engineering. This suggests AI is becoming an established source of corporate cyber risk, rather than being viewed simply as a future threat.
The first line of defence
At its heart, shadow AI is a human challenge. It’s usually done not out of malice or negligence but simply staff trying to work faster and more productively when they don’t have the approved tools available to support them. Only a fifth (22%) of organisations make employee awareness a priority.
Over a third (35%) of respondents say that using GenAI without permission is now the most common employee mistake of the past 12 months. A further 30% say the same about leaking confidential information via an AI tool. Lack of employee awareness around information security challenges (39%) is among the top challenges cyber leaders face today.
The answer for many is to put in place clear guidelines to ensure employees use approved AI tools; and do so safely and responsibly. Over half (51%) of respondents say they have updated AI usage policies to set clear parameters for staff in order to prepare for emerging threats; the top cited answer. Over two-fifths (42%) claim to have a formal, enforced AI policy in place, while 43% say staff are trained on responsible and secure AI use.
It appears that there’s still not enough thought going into establishing the right governance frameworks prior to AI adoption. Many organisations admit to having relaxed their controls for AI in order to keep pace with adoption.
To what extent do you agree: We adopted AI too quickly and now face challenges scaling it back or using it more responsibly
But is this enough? Respondents at organisations of 250+ employees are more likely than those with under 100 employees to report a formal, enforced AI policy (46% vs 36%), but no less likely to say staff have built AI into workflows without the security team knowing (24% vs 21%).
A majority (52%) of organisations also say they adopted AI too quickly; almost the same share as last year (54%). It appears that there’s still not enough thought going into establishing the right governance frameworks prior to AI adoption. Many organisations (17%) admit to having relaxed their controls for AI in order to keep pace with adoption, rising to 21% in the US.
The exact balance between productivity and security will always be down to the individual organisation and dependent on its specific risk appetite. But it remains somewhat disappointing that, so few are prioritising AI usage policies (22%) and employee security awareness and behaviour (22%) over the coming year. AI governance matters now more than ever.
Which of the following apply to AI in your organisation today?
