knowledge base · agents

AI agents, mapped.

From chunking to guardrails: how a modern agent is built, with the data pipeline (RAG), the action loop (frameworks, tool calling, MCP) and the evaluation mesh that decides whether it can ship.

$ click a card for details · ctrl+k to search

learning map

Topics are laid out by level, from foundations to advanced. Arrows point from a prerequisite to what builds on it. Select a topic to preview it, or follow the highlighted next step.

Your next step

0 of 15 studied

start this topic

01 Foundations

The ideas everything else builds on.

02 Practitioner

Apply the controls and make them work in practice.

03 Advanced

Design, scale and operate at depth.

studied
ready to study
prerequisites missing
arrows go from a prerequisite to what builds on it

Your next step

LLM - large language model

The LLM is the agent's generative engine: it receives context (system prompt, history, retrieval, tools) and produces the next response. It does not query a fact base; it uses the context to look like it does.

concept · 4 min read

key points

query external data on its own: without retrieval or a tool, it only knows what was in training
finite context window: go past the limit and the history disappears; go past the limit and the cost of the view changes.

prerequisites

No prerequisites: a good place to start.

opens the way to

relations

injects context
is the core
filters input/output

AI pipeline

From the raw document to the guardrail metric: the path of data and quality. Each chip opens the matching topic.

01ingest
documents become retrievable pieces
chunking
gate: each piece fits in the context and preserves the local idea
02index
pieces become vectors and enter the database
embeddings vector-db
gate: vectors persisted and queryable by similarity
03retrieve
the right piece for the right question
retrieval
gate: stable ranking and coverage of the relevant passages
04act
the agent decides, calls tools, answers
agent frameworks tool calling orchestration
gate: typed, audited action within the agent's scope
05evaluate
measure so you can trust
evals benchmarks LLM as judge regression evals
gate: measured quality, protected baseline, gate on merge

topics

RAG - retrieval-augmented generation

RAG combines search and generation: the model only answers after pulling the relevant passages from an external base, so the answer is anchored in material that exists and can be cited.

Chunking is the first technical decision of RAG: the size, the overlap and the cutting strategy determine what retrieval can find later. Bad pieces become answers without context.

technique· stage: ingest

Embeddings map text to a space where distance becomes semantics: 'dog' and 'pet' end up close, 'dog' and 'satellite' end up far. It is the engine of RAG's similarity search.

technique· stage: index

A vector DB is the component that stores embeddings and answers 'which vectors are closest to this one?' in milliseconds. It is where ANN (HNSW, IVF) pays the cost of semantic search.

component· stage: index

Retrieval is where search becomes an answer: dense, sparse, hybrid and reranking decide whether the context reaching the LLM is useful or junk disguised as evidence.

technique· stage: retrieve

LLM - large language model

The LLM is the agent's generative engine: it receives context (system prompt, history, retrieval, tools) and produces the next response. It does not query a fact base; it uses the context to look like it does.

Agent frameworks

An agent framework is the skeleton that holds the agent's loop: state, plan, tools and what happens between each call to the model. Without a framework, the agent becomes a script full of if/else that nobody can audit.

component· stage: act

Tool calling is the contract between the LLM and the world: the model returns structured JSON (name + arguments), the runtime validates it against a schema and executes. It is what turns text into action.

technique· stage: act

Orchestration is the pattern above the single agent: pipeline (fixed sequence), supervisor (one agent commands N) or swarm (N agents self-organize). It defines who talks to whom, when and in what order.

component· stage: act

MCP - model context protocol

MCP (Model Context Protocol) standardizes how an LLM client discovers, lists and calls capabilities exposed by servers: tools (actions), resources (data) and prompts (templates). A single interface to integrate with any source.

Evals - quality measured in production

Evals are the harness that measures whether the answer is good: a dataset of examples, a metric (faithfulness, relevance, groundedness) and a runner that executes end to end. Without evals, improving an agent is guesswork and every tweak becomes routine.

component· stage: evaluate

Benchmarks - public reference

Benchmarks (SWE-bench, GAIA, τ-bench, HumanEval) give a shared reference: you run the agent on the same dataset others published and the number speaks the same language. Classic risk: optimizing for the benchmark and losing in the real world.

technique· stage: evaluate

LLM as judge - automated rubric

LLM as judge automates grading: a model applies a rubric (pairwise, Likert, score) to the system's output and returns a scalable verdict. It works when properly calibrated; it becomes circularity when the judge uses the same model it is judging.

technique· stage: evaluate

Regression evals - fixed suite on the PR

Regression evals are the immutable suite: closed dataset, gate on merge, comparison against a baseline. It is what turns 'I think it got better' into 'metric X dropped 2 points, block the PR'.

component· stage: evaluate

Guardrails - input and output filters

Guardrails validate input (prompt injection, PII, jailbreak) and output (PII, schema, policy) on every turn. They complement evals: they hold back what goes to the user, but do not measure whether the answer is good.