Research subscriptionsTechnology providersCareersTrust Center

Cybersecurity and risk

AI security: protecting generative AI and AI agents

AI security covers the risks specific to systems built on machine learning and large language models, such as prompt injection, leakage of sensitive data, poisoned training data and agents taking unintended actions.

Reviewed by Krynthnode Research. Last updated .

Why it matters

Generative AI is being connected to company data and tools faster than security teams can review it. The OWASP Top 10 for Large Language Model Applications lists prompt injection as the leading risk.

What to get right

  • Keep an inventory of AI systems in use, including features embedded in vendor products.
  • Treat everything a model reads, including documents and web pages, as untrusted input.
  • Give AI agents the minimum permissions they need, and require approval for consequential actions.
  • Log prompts, outputs and actions so incidents can be investigated.

Common questions

What is prompt injection?

An attack in which instructions hidden in content the model processes, such as an email or a web page, cause it to ignore its intended instructions or leak information.

Can existing security tools protect AI systems?

Partly. Identity, data protection and logging still apply, but AI-specific testing and controls are needed for model behaviour.

Our research on this topic

The Monday Brief

A concise weekly briefing on technology, cybersecurity and regulation, delivered every Monday.

Subscribe by emailFollow on LinkedIn