Skip to content
sec/atlas
back to the map
concept
4 min read

LLM - large language model

The LLM is the agent's generative engine: it receives context (system prompt, history, retrieval, tools) and produces the next response. It does not query a fact base; it uses the context to look like it does.

OpenAIAnthropic ClaudeGoogle GeminiMistral

What it is

An LLM (Large Language Model) is a generative model trained to predict the next unit of text (token) given everything that came before. Instead of 'knowing' facts like an encyclopedia, it learned patterns of language, code, reasoning and world knowledge embedded in the training data.

When you send a question, the model does not look anything up: it reads the entire context you handed it (system prompt, history, retrieval, tool outputs) and continues the pattern. The quality of the answer depends almost entirely on what is in that context, not on what the model 'knows'.

Tokens: the model's real unit

Models do not read words, they read tokens: pieces of text that can be a whole word, part of a word, a number or a symbol. A paragraph in Portuguese typically becomes 1.3x more tokens than words. This matters because everything is counted in tokens: input prompt, output response, cost, latency and the limit.

The context window is the maximum size the model can look at at once, measured in thousands (K) or millions (M). Go past the limit and the start of the dialogue ceases to exist for the model. That is why retrieval, summarization and history trimming exist as techniques.

approximate token counttext
  'Olá, mundo'                     -> 3 tokens  ('Ol', 'á', ', mundo')
  'Grandes modelos de linguagem'    -> 7 tokens
  'functionCalling'                 -> 6 tokens  (3 words + 3 subwords)

  rule of thumb PT-BR:  ~1 token per 0.75 word
  rule of thumb EN:    ~1 token per 0.75 word

System prompt: the contract with the model

The system prompt is the system message that defines the model's behavior: role, constraints, output format, refusal policy. It is the only part of the context that the ordinary user does not write, and that is why it is the most valuable.

A good system prompt describes three blocks: identity and purpose, hard rules (what the model must never do) and output format (how to respond). The more specific it is, the less the model invents.

minimal system promptmarkdown
You are a technical support agent for company X.

Rules:
  - answer in Brazilian Portuguese, professional and direct tone
  - if you do not know the answer, say 'I don't know' and offer to open a ticket
  - never invent prices, deadlines or ticket numbers

Format:
  - answers in at most 3 paragraphs
  - cite the knowledge base article when used (format KB-[IDX])

Cost and latency

Each call to the model has two cost components: input tokens (what you send) and output tokens (what it generates). Output tokens usually cost 3 to 5 times more than input tokens in commercial APIs, because generation is serial and more expensive for the provider.

Latency follows two simple rules: the time to the first character depends on the size of the prompt, and the total time depends on how many tokens the model generates. That is why large prompts take long to start and long answers take long to finish, and why 'be concise' in the system prompt saves real money.

estimated cost per calltext
  prompt:      2,000 input tokens
  response:    400 output tokens

  price (generic example):
    input   $3 / 1M tokens  -> $0.006
    output  $15 / 1M tokens -> $0.006
    total                      $0.012 per call

  at 10,000 calls/month on the same pattern: ~$120/month

What the model does not do

  • query external data on its own: without retrieval or a tool, it only knows what was in training
  • remember past conversations: between calls the model is stateless; each turn starts from scratch
  • execute code: the model returns strings, including strings that look like code; execution stays with the runtime
  • guarantee factual truth: hallucinations are the default state when the model completes patterns without anchoring in a source
  • keep secrets by itself: secrets in the system prompt do not stay secret forever; the prompt leaks via exfiltrated logs

Limitations

  • finite context window: go past the limit and the history disappears; go past the limit and the cost of the view changes.
  • the model is not deterministic: the same question can generate different answers on each call; temperature controls this
  • outdated knowledge: training has a cutoff date; what changed afterwards does not exist for the model
  • cost grows with use: agents that call the LLM several times per task multiply the cost, times the usage
Context windowcontext windowThe maximum number of tokens the model can read at once: the prompt, the entire concatenated history and the output it will generate. It is the LLM's short-term memory. Huge windows seem to solve RAG, but they amplify cost, latency and the risk of the model getting lost in the middle of the text, so good context is small, well-chosen context.see in glossary →

connections on the map

updated on 14/10/2026 · sources to go deeper · report an error

Unlocks

test what you learned

Loading questions…