Back to writing

The prompt is not a security boundary

When an LLM writes the query, authorizing who asks isn't enough. What matters is what the answer can touch.

In systems that generate SQL, filters or code with a language model, there are two authorization questions, and it's easy to answer only the first:

  • Who can ask for the query?
  • What can the generated query touch?

Authorizing the first and ignoring the second is locking the door and leaving the window open. The user was allowed to ask about their own unit's data; the SQL the model wrote doesn't know that.

Load only what was already authorized

The strongest protection against the model reaching what it shouldn't is for the data not to be there. Instead of opening an engine with every context and trusting the model to pick the right one, the run receives only what already passed authorization:

ctx = authorize_data_context(user, context_id)  # server-side predicate
engine = load_engine_only(ctx.storage_prefix)

If the process for that run can't see the other datasets, no prompt, however creative, gets the model to them.

A system prompt is a suggestion

“Only use these tables” in the system prompt isn't a control. The model is a probabilistic component and will, eventually, do something else, whether by mistake or because someone asked nicely. The output has to pass deterministic validation before it runs:

question
  -> authorized context
  -> model drafts a candidate
  -> parser / AST / validation
  -> reject if invalid
  -> audit
  -> run on a scoped engine
  -> answer

The minimum guard

For generated SQL, the minimum I consider acceptable:

  • a single statement;
  • read-only, with DDL, DML and admin commands blocked;
  • no file or network functions;
  • every referenced relation inside the context's allowlist;
  • row limit, timeout and memory budget;
  • a read-only connection, even if the guard fails.

The last item is what lets you sleep: the guard is one layer, the read-only connection is another and the scoped engine is a third. None of them has to be perfect on its own.

Regex is scaffolding

You can start with regexes, as long as adversarial tests cover them. But a regex is a temporary floor, not a destination. Comments, strings, quoted identifiers and dialect-specific syntax make any regex fragile:

SELECT * FROM "public"."Patients" /* ; DROP */
SELECT * FROM report WHERE note = 'FROM secrets'

A parser that speaks the dialect the query will run on turns the question “does this look dangerous?” into “which relations does this tree reference?”, and the second question has an exact answer.

Injection comes from the data

The user's question isn't the only untrusted input. Column names, cell values, spreadsheet metadata, text from an ingested PDF: all of it reaches the model's context and may contain something that looks like an instruction. Delimiting those elements as data helps, but the decisive defense is still validating the output. If the answer can only be a SELECT over authorized relations, an instruction hidden in a spreadsheet has nowhere to go.

Fewer tools, smaller surface

On the sensitive query path, the agent gets only what it needs. If the task is analytical reading, there's no filesystem writing, arbitrary network, shell, free-form Python, database mutation or another service's credentials. Least privilege applies to agents just as it does to people, and more so: the agent reads everything put in front of it.

This goes hand in hand with Mask before you call the LLM: one layer handles what the model sees, the other handles what it can do.