AI Security 101
Plain definitions for the terms that come up when enterprises try to secure AI. Written by the team at Mavs AI.
Agent Drift
Agent drift is an agent moving away from the goal it was given as it works, so that its later steps no longer serve the task it started on. It can build up on its own, since each step is taken on the basis of what the last one returned, or be caused from outside by instructions hidden in content the agent retrieves. This is a different problem from model drift, which plays out over weeks rather than inside a single task.
AI Control Layer
An AI control layer sits between the people, applications and agents on one side and the models they call on the other, so that every request passes through one point where security can be applied. It has three parts: the policy that decides what data may reach which model, the enforcement applied to live traffic, and the record of what was sent and what was changed.
AI Guardrails
AI guardrails are the controls placed around a model to constrain what reaches it and what it returns. They fall into three groups. Input guardrails act on the request, and cover sensitive data protection, prompt injection and jailbreak detection, and topic or acceptable-use limits. Output guardrails act on the response, and cover content safety, groundedness, format conformance and data leakage. Action guardrails decide whether an agent may carry out a given action.
Jailbreak
A jailbreak is an attempt to talk a model past its own restrictions so it produces something it was built to refuse. It is closely related to prompt injection and often grouped with it, the difference being the target: injection overrides the instructions the application gave the model, while a jailbreak works on the limits set by the model provider.
Model Drift
Model drift is a model's accuracy falling over time because the data it meets in production no longer matches the data it was trained on. It takes two forms: the inputs themselves change, or the relationship between the inputs and the right answer changes. Neither announces itself, so a model can keep producing confident output long after it has stopped being reliable.
Model Poisoning
Model poisoning is an attack that corrupts a model rather than a request, by tampering with its training data, its fine-tuning data or its weights so that it behaves the way the attacker intends. The effect is built into the model, so it carries across every request and is not visible in the prompt. It differs from prompt injection, which manipulates a model while it is running and leaves the model itself unchanged.
Prompt Security
Prompt security is the protection of the prompt itself, treated both as a channel that carries sensitive data out of the organisation and as an attack surface that can be used to manipulate the model. The first half is about what the prompt contains: personal, regulated and business-sensitive data, assembled from what the user types, what the application adds and what sits in any attached file. The second half is about what the prompt can do: prompt injection and jailbreak attempts that try to override the instructions the model was given.
Prompt Injection
Prompt injection is an attack that uses the input itself to make a model follow the attacker's instructions instead of the ones it was given. It is direct when the attacker is the user typing into the model, and indirect when the instructions are hidden in content the model pulls in, such as a web page, a document or an email, in which case the user need not know it happened. OWASP has kept it at the top of its Top 10 for LLM applications as LLM01.
Runtime Agent Security
Runtime agent security is the enforcement of security controls on an AI agent as it works, applied to each step it takes: the prompts it sends, the tools it calls and the data it moves. An agent is given a goal rather than a script, so its steps are decided as it goes and the permissions set beforehand do not describe what it will do. Controlling it means checking intent step by step, since the same action can serve the goal or drift from it, and content the agent retrieves can redirect it mid-task.
Runtime Prompt Security
Runtime prompt security is the enforcement of security controls on a prompt in the request path, at the moment it is sent. A prompt is put together only when the request is made, out of what the user types, what the application injects, what a retrieval step pulls in and what sits in an attached file, so that is the earliest point at which there is anything to check. A control written at build time, such as pre-deployment red teaming or a hardened system prompt, is set before there is anything to inspect. A control that watches data at rest, such as file classification and labelling, never sees the request at all. The assembled prompt is the first point at which what is actually being sent can be read and acted on.
Runtime Security
Runtime security is the enforcement of security controls while software is executing, rather than at build, deployment or configuration time. The object being protected is whatever is live: a process, a workload, a container, or a request. Applied to GenAI, the live thing is the model call, so runtime security means acting on the prompt or the agent action as it happens, which matters more here than in conventional software because a model's behaviour is not deterministic and cannot be established before it runs.
Secure Gateway
A secure gateway is a single point that traffic passes through on its way to an external service, so that authentication, policy and logging are applied in one place rather than configured on every client. For GenAI, a secure gateway sits between an organisation's users, applications and agents and the model providers they call, which is what makes it possible to apply one policy across several models and to keep one record of what was sent.
Shadow AI
Shadow AI is the use of AI tools inside an organisation without the knowledge or approval of the people responsible for security and compliance. It tends to grow where the sanctioned tools are missing or blocked: someone with work to do moves to a personal account or an unapproved app, where the organisation has no policy, no logging and no way to know what data was sent.
Synthetic Data
Synthetic data is artificially generated data that resembles real data closely enough to be used in its place, most often for training and testing where the real data is restricted. In prompt security, sensitive values in a live request are replaced with synthetic stand-ins before the model sees them, and the real values are restored in the response. The model can still do its job because the stand-ins behave like the real values, so the answer comes out the same.
