bryan@devbox ~ % cat post.md

Claude Certified Architect Exam: Domain 4 — Prompt Engineering

The quality of an model’s output depends largely on the quality of its input. That sounds obvious enough to be useless, so this piece is about the specific part: which input choices actually move the output, and why each one works.

The techniques below are roughly ordered from the ones that matter on every prompt (clarity, context, structure) to the ones that matter once you’re running prompts in production (structured output, quality control, evaluation).

Clarity and Specificity

There are multiple reasons to avoid ambiguity. The obvious reason is if an instruction can be read more than one way, the model may read it in a way you didn’t intend. The less obvious reason is that a model may recognize the ambiguity and spend extra computation and time working out the best interpretation. Even when it lands on the right reading, you’ve spent time and tokens that a precise instruction would have saved.

Negative Framing

You know how “don’t think of a pink elephant” makes you do exactly that. Something similar happens with language models. Negation gets diluted in context, and the model does the thing you told it to avoid. So phrase the instruction positively. Instead of “don’t use abstract clichés,” write “use concrete examples.” Sometimes there’s no clean opposite to point at. Two ways around that:

  • Consequence framing: “if third-party libraries are included, the task will be marked as a failure.”
  • Correct/incorrect examples: show the failure mode next to the version you want.

The System Prompt

The system prompt is easy to overlook and does more work than any individual user prompt. It’s the place for everything that stays constant across the session, in this order:

  1. Role
  2. Overall context
  3. High-level instructions
  4. Constraints (things to avoid, topics that are off-limits, other limits)
  5. Examples

The order matters. So does length: the system prompt is resent with every request, so every token in it is a token you pay for on every call. Clarity and conciseness pull against each other here, and the balance is worth tuning deliberately.

Including Examples

Detailed examples are powerful but aren’t always necessary. For common, basic tasks, a clear structured prompt is usually enough. Examples become essential when the exact output format matters, or when the task involves taste or judgment that’s easier to show than describe.

How many depends on how much your inputs and outputs vary. When expected outputs are consistent, one example does it. When there’s real variance, three is a good target.

Chain of Thought

Chain of thought means instructing the model to show its reasoning, sometimes as simple as “think step by step.” It helps most on multi-step logical tasks and non-obvious trade-off decisions, because it stops the model from trying to produce an answer in a single leap. Writing out the reasoning steps gives the model a chance to catch its own errors along the way.

Extended Thinking

Extended thinking is related to chain of thought but works differently. The reasoning happens in a separate channel, kept out of the visible response, and it typically runs on a reasoning model, so the reasoning itself tends to be more efficient.

One common misconception is worth noting: extended thinking is still billed. Thinking tokens count as output tokens. You’re paying for better reasoning, at normal output rates.

The real structural advantage is control. Extended thinking exposes an effort level you can dial up or down per request; prompted chain of thought has one mode, on.

Which to use:

  • Extended thinking when the task is genuinely hard, or when you don’t know in advance how hard it will be.
  • Chain of thought when you need to dictate the specific steps, because you have an actual checklist you want followed.

Role and Persona

You’ve seen role assignment before: “act as an expert in contract law.”

Telling a model to act as an expert doesn’t grant it knowledge it doesn’t already have. What it does is shape behavior and focus the model on knowledge that’s already present. It steers vocabulary, reasoning depth, and how the model scopes the task.

Role and persona are different levers. Role defines functional expertise. Persona is a stylistic lens: tone of voice, rhetorical style.

XML Tags for Complex Prompts

Some prompts don’t decompose cleanly into separate calls. When a single prompt has to carry several components at once, XML tags separate the instruction from the context, the examples, the output format, and so on.

There’s a second benefit that’s easy to miss: it reduces prompt injection risk. Without clear delimiters, a model can mistake an example inside your prompt for data it’s supposed to process.

There’s no required tag vocabulary. Pick names that describe what they wrap and use them consistently.

Structured Output

When a prompt’s output feeds another system, whether that’s another prompt or deterministic code, you usually want structured data, typically JSON.

Claude lets you define a precise JSON schema for the output to conform to, and the response is constrained to match it. (The constraint holds for completed responses; a refusal or a response truncated at the token limit is still something your code should handle.) The schema defines fields, data types, constraints, and which fields are required versus optional.

Be careful about which fields you mark required. If a field is required but a given input genuinely has no value for it, the model will invent one. A required field is an instruction to always produce a value, and the model follows it.

Quality Control

Quality control for non-deterministic systems is a different discipline from testing code. You’re checking for entirely new classes of problems.

The best known is hallucination. Some things that help:

  • Don’t instruct the model “don’t hallucinate.” It doesn’t work, and it’s negative framing besides.
  • Schema constraints, as described above.
  • Require citations for factual claims. The caveat: you then need a step to verify the citations are real.

Citations are one form of grounding. Others include supplying the source document directly, or giving the model access to a vector database so it can pull relevant sources on demand.

Sometimes a validation loop is the right tool, and negative constraints are a good example. The validator checks for the condition you’re trying to avoid and re-prompts the model to correct its output, repeating until the output passes or the retry limit is exhausted.

Prompt Evaluation

Hallucination prevention and grounding are quality control for the output. Prompt evaluation is quality control for the prompt: testing prompts systematically, the way unit tests evaluate code.

It works by running prompts against a labeled dataset. The pass criteria for an individual case depend on your use case, but the metrics that matter across the suite are consistent:

  • Pass rate
  • Regression rate
  • Latency
  • Cost per eval

Tests can be code-based or model-based. A useful suite covers typical cases, edge cases, and adversarial cases, with enough volume that a pass means something statistically.

Where This Leaves You

Most of the gain from prompt engineering comes from the unglamorous end: being specific, supplying context, framing positively, and structuring what you send. The techniques further down this list (structured output, validation loops, evaluation suites) are what you add when prompts stop being something you write once and start being something you maintain.

A prompt you’ll run twice deserves clarity. A prompt running in production deserves an eval suite.

Next up: Domain 5, Context Management