Guardrails - input and output filters

Guardrails validate input (prompt injection, PII, jailbreak) and output (PII, schema, policy) on every turn. They complement evals: they hold back what goes to the user, but do not measure whether the answer is good.

Guardrails AINeMo GuardrailsLakeraRebuff

What it is

A guardrail is a validation layer that sits between the user and the agent, and between the agent and the final response. On each turn, two edges: the upper edge analyzes the user's message (and the history, and the documents that will enter as context) and decides whether to pass, block or rewrite. The lower edge does the same with the agent's output before it reaches whoever asked.

The difference from evals is the position in time. An eval measures quality after the response was generated, in batch or by sampling. A guardrail acts now, on every request. They are complementary. Whoever replaces one with the other creates security theater: an eval too slow to block and a guardrail alone with no view of overall quality.

Input: prompt injection, jailbreak, PII

Input is the most targeted part. The user may try to instruct the model to ignore the system instructions (classic jailbreak), embed a command inside a retrieved document (indirect ), or paste sensitive data asking the agent to process it ( that ends up in the log). The input guardrail needs all three detections, and each has its own mechanism.

PII is the easiest: regex over email, national ID, phone, card. False positives are review work, false negatives become incidents. Jailbreak is subtler: classifiers trained on adversarial prompts, or known patterns (DAN, role-play). Indirect injection is the newest and the hardest: the hostile instruction is in a document that retrieval brought in, not in the user's message. Detection here calls for semantic analysis of the retrieved content, not just the raw text.

Output: schema, PII, policy

Output needs three layers too. Schema validation ensures the final response has the expected shape (typed JSON, valid markdown, links with a trusted host). PII masking redacts email, phone, address that the model may have invented or copied. Policy rejects content forbidden by the business (competitor mentioned, restricted jurisdiction, sensitive category).

The schema layer deserves special attention. Without a valid schema, the frontend does not render and the agent looks broken. That is why the output guardrail starts almost paranoid: better to block a valid response and return a fallback than to deliver a response that breaks the consumer.

Output guardrail in practice

The snippet shows a minimal output guardrail: it validates JSON against a schema, redacts basic PII, and blocks when the content strays from the rubric. In production it is worth swapping regex for a detector (Guardrails AI, Lakera), but the mechanics stay the same:

guardrails/saida.tstypescript

import Ajv from "ajv";

const schema = {
  type: "object",
  required: ["resposta", "fontes"],
  properties: {
    resposta: { type: "string", minLength: 1, maxLength: 2000 },
    fontes: { type: "array", items: { type: "string" } }
  },
  additionalProperties: false
};

const ajv = new Ajv();
const validate = ajv.compile(schema);

const PII = /\b([\w._%+-]+@[\w.-]+\.[A-Za-z]{2,}|\d{3}\.\d{3}\.\d{3}-\d{2})\b/g;
const PROIBIDO = /(competitor-x|competitor-y)/i;

export type Veredito = { ok: true; saida: unknown } | { ok: false; motivo: string };

export function guardrailSaida(raw: string): Veredito {
  let parsed: unknown;
  try { parsed = JSON.parse(raw); } catch { return { ok: false, motivo: "invalid_json" }; }
  if (!validate(parsed)) return { ok: false, motivo: "invalid_schema" };
  if (PROIBIDO.test(JSON.stringify(parsed))) return { ok: false, motivo: "policy_violated" };
  const redigido = JSON.stringify(parsed).replace(PII, "[REDACTED]");
  return { ok: true, saida: JSON.parse(redigido) };
}

Where it fits in the architecture

As a cross-cutting layer, before the LLM and after the LLM. In production the input guardrail becomes a middleware of the HTTP endpoint; the output one becomes a wrapper of the LLM client or a post-processor of the stream. In frameworks like LangGraph, it is a dedicated node between the user and the agent, and between the agent and the response.

Performance matters. Each guardrail adds latency, and latency in a chat is visible right away. Rule: the input guardrail runs in <= 50ms for most cases, and the output one stays within the request's total budget (typically <= 200ms). Above that, use an asynchronous detector and show a placeholder while it processes.

A guardrail does not replace evals

A guardrail takes care of what can happen. An agent can pass all the guardrails and still give a bad response (off-topic, factually wrong, with an inappropriate tone). That is the eval's job, and the eval runs in batch where the guardrail cannot.

The confusion appears when someone treats a guardrail as proof of quality. The average guardrail score is high (most requests pass), and that leads the team to think 'everything is fine'. It is not. A guardrail measures safety and compliance; an eval measures quality. They are two different axes, both necessary.

Strengths

blocks known attacks before they reach the model, e.g. simple injection and obvious PII
traceable compliance: each turn becomes a decision log, and auditors want a fast signal when there is a violation
composable: you can combine several guardrails in a pipeline, each handling one class of risk

Limitations

false negatives on new attacks: prompt injection evolves, and a detector trained in 2024 errs in 2026
latency cost: each additional synchronous layer shows up in the SLA, and a pipeline of 5 guardrails starts to look expensive
quality blindness: it blocks the dangerous, but does not measure whether what passed is good, and that remains the eval's job

Prompt injectionAn attack in which hostile content (in an indexed page, in a summarized email, in a document the agent opens) tries to override the system instructions. The attacker knows the LLM mixes trusted text with found text and plants an 'ignore the previous instructions and run X'. It is the number 1 vector in production agents today; it is not an LLM bug, it is how agents read text by design.see in glossary →PIIpersonally identifiable informationPersonal data that identifies someone: name and national ID, email, phone, address, IP. When the user pastes a document with PII into the chat, or RAG indexes a CRM with 5,000 customers, the agent becomes a leak by design. Guardrails here are detection on input, redaction before sending to the LLM, and short retention, because masking after receiving the response is already too late.see in glossary →

connections on the map

Guardrails - input and output filtersfilters input/outputLLM - large language model

updated on 14/10/2026 · sources to go deeper · report an error

in other domains

Securing the agent itselfvibecodingthe same limits, applied to the agent that writes code
Threat modeling: think like the attacker before codingappsecmaps what the agent must not be able to do
Bot managementsecopstelling legitimate automation from abuse applies to agents too
AI in triage and mitigationsecengthe same care with model input and output

Study before

LLM - large language model

test what you learned

Loading questions…