In the spring of 2023, a mid-sized regional hospital network deployed an LLM-based pipeline to automate medication reconciliation, the process of comparing a patient's medication orders against every other list of medications the patient is known to take, catching discrepancies before they become adverse events. The input to the system was not clean. It was never going to be clean. Medication histories arrive at a hospital from a dozen sources simultaneously: handwritten intake forms photographed and OCR'd into broken strings of text, free-form nursing notes typed at 2 a.m. into an underfunded EHR system, faxed discharge summaries from other facilities whose formatting conventions belong to a different decade, and patient self-reports transcribed by whoever had a free moment at the front desk. A single patient's medication history might look like this:
The engineering team faced a decision that felt, at first, like a formatting question. It was not a formatting question. It was an architectural question, and getting it wrong had clinical consequences.
Their first attempt was zero-shot: send the raw text to the model with a prompt asking it to return a JSON list of medications with fields for drug name, dose, frequency, route, and any flags. For clean inputs — the kind that appear in textbook examples and vendor demos — this worked acceptably. For inputs that looked like the one above, it did not. The model normalized aggressively. It resolved ambiguity silently. "The little pink one for cholesterol" became rosuvastatin, 10mg, QD — a specific dose fabricated from genre conventions, because rosuvastatin starter doses are often 10mg and the model had seen that pattern ten thousand times. The warfarin hold instruction disappeared entirely in two out of five test cases, absorbed into the background noise of the prompt. Zero-shot extraction, applied to genuinely messy clinical text, does not fail loudly. It fails quietly, with confident, well-formatted JSON.
The team's second attempt introduced few-shot examples and a chain-of-thought instruction: "Before producing the output, reason through each medication mentioned, note any ambiguities, and then produce the structured list." This helped. The model now surfaced the rosuvastatin uncertainty rather than resolving it silently. The warfarin hold instruction survived more reliably because the reasoning step forced the model to enumerate every medication-adjacent statement before committing to output.
But before understanding why that worked, it helps to feel the problem it was solving. The team had watched the warfarin hold instruction disappear — not because the model couldn't parse it, but because the model had processed the entire note and collapsed it into a structured record in a single operation, the way a tired reader skims a long email and misses the one action item buried in paragraph four. What they needed was a way to force the model to slow down — to enumerate every claim in the note before it was allowed to format any of them. The chain-of-thought instruction was that mechanism.
But the team wanted more. They wanted downstream systems — pharmacy verification software, clinical decision support tools, the EHR write-back module — to consume the output programmatically. That meant strict schemas. What they needed was not just a format — a list of required fields — but something closer to a small language: a system with its own vocabulary of valid values, its own grammar for how those values could be combined, and its own rules for what counted as a well-formed record. In software engineering, this is called a Domain-Specific Language: a structured notation built for one specific task, enforced not by a compiler but by the prompt itself. So they designed a Meta Language: a DSL that defined exactly how medication records should be represented, with required fields, enumerated values for frequency codes, a controlled vocabulary for flags, and explicit syntax for expressing uncertainty. The schema was careful and complete. It was also, under certain conditions, a trap.
When the team tested the Meta Language schema on a new batch of intake notes, they found a failure mode they had not anticipated. The model, trained implicitly to produce well-formed outputs and now given an explicit, rigid schema to conform to, began prioritizing format compliance over semantic accuracy. A note that read "patient unsure of dose, takes 'half a pill' of amlodipine" was rendered as frequency: "QD", dose: "2.5mg" because 2.5mg is half of the standard 5mg tablet, and the schema required a numeric dose, and the model found a number that fit. The uncertainty had been laundered into a confident structured record.
In machine learning, a model that memorizes the surface features of its training examples rather than their underlying logic is said to overfit. What happened here is the same failure applied to format: the model had learned that a well-formed output satisfies the schema's surface requirements, and it optimized for that surface at the expense of the meaning the schema was designed to capture. This is format overfitting. The JSON validates. The downstream system accepts it. A pharmacist reviewing a clean structured record has no reason to suspect that the number in the dose field was inferred from the geometry of a pill the patient couldn't name.
Three architectural options were available to the engineering team. Each made a different bet about what the model needed in order to behave correctly. Each failed differently when that bet was wrong. The question this chapter is built to answer: given a realistic extraction task where inputs are messy, ambiguity is genuine, and downstream consumers need structured data, how do you choose between zero-shot, few-shot with chain-of-thought, and a Meta Language schema — and how do you design each option so that its failure mode is visible, recoverable, and does not masquerade as success?
To understand why prompt structure is an architectural decision, you need a precise picture of what happens when a language model processes a prompt. A language model does not read your prompt the way a person reads a paragraph — accumulating understanding, pausing to reflect, revising earlier interpretations as new sentences arrive. It performs a single sweep through all the tokens in the context window simultaneously, computing at the end of that sweep a probability distribution over which token should come next. That sweep is called a forward pass. It happens once per token generated, and once a token is chosen, the model cannot revise it — there is no internal editor.
Generating text is therefore a sampling process: at each step, the model computes a distribution over all possible next words and selects from it. What determines those probabilities? Everything that came before in the context window — including the prompt you wrote. Change the prompt, and you change the probability distribution the model samples from. Change that distribution, and you change what the model is capable of reasoning about. This is what it means for prompting to be architectural: the prompt is not a command. It is the environment in which the model's inference unfolds.
You have already seen this. The same model, given the same clinical note, dropped the warfarin hold instruction under zero-shot and surfaced it under chain-of-thought. The only thing that changed was the prompt. That is what it means for the prompt to be the computational environment.
In a zero-shot prompt, the model receives the task description and the input and nothing else. The model must infer everything — what level of detail is expected, how to handle conflicting information, what to do when the input contains a phrase like "the little pink one" — from the statistical regularities it absorbed during training.
When you write a zero-shot prompt asking for a JSON medication record, you are creating a context in which the tokens {, "drug_name", ":", "metformin" are very probable next tokens, because that pattern appears overwhelmingly in the model's training data whenever a structured extraction prompt appears. The prompt does not instruct the model to fill in a plausible dose. It makes filling in a plausible dose the most probable thing to do next. The difference is not semantic. It is the difference between a command and a gravitational field.
This is why zero-shot fails silently. The failure is not a hallucination in the colloquial sense — the model inventing something fantastic and obviously wrong. It is something more dangerous: a plausible confabulation, a value that fits the schema and fits the genre conventions of medication dosing and is therefore invisible to any downstream system that does not know to look for it. The output validates. It simply is not true.
Unlike the training process — in which the model's weights were adjusted over billions of examples — inference involves no weight updates at all. The model's weights are fixed. What changes when you change the prompt is the context those fixed weights operate on. This is sometimes called in-context learning: the model behaves differently based on what's in the context window, without any underlying change to what it has learned. It is adaptation without memory.
An example in a prompt is not a demonstration the model watches — it is a piece of the context window that shapes which token sequences are probable. Add an example of a model surfacing uncertainty, and you have made surfacing uncertainty more probable. Remove it, and you have made resolving uncertainty more probable. The examples are not there for the model to learn from. They are there to be there.
Chain-of-thought prompting addresses premature compression by making the reasoning process part of the output. Because every output token is conditioned on all preceding tokens — including the reasoning trace — a reasoning step that explicitly names an entry makes it improbable for the output to omit it. The model would have to generate output tokens that conflict with its own immediately preceding context. The reasoning text that appears before the final output is not decorative. It is a form of externalized working memory: a sequence of intermediate token commitments that forces the model to surface and retain information it would otherwise compress away.
The few-shot examples — called exemplars in the prompting literature to distinguish them from illustrations — are not there to show the reader how extraction works. They are there to show the model what the generalization target is. Inputs that don't resemble any exemplar fall back to zero-shot behavior. The exemplar selection is therefore an architectural decision with direct consequences for which failure modes the system inherits. Examples closer to the new input tend to exert stronger influence on the output; place your most representative ambiguous case last in the exemplar sequence, immediately before the new input.
A Meta Language, in the context of prompt engineering, is a structured schema embedded in the prompt that defines a formal vocabulary, syntax, and set of rules for the model's output. It is, functionally, a Domain-Specific Language defined not in code but in natural language or structured notation, enforced not by a compiler but by the statistical pressure the prompt exerts on the model's output distribution.
What the Meta Language actually does, at the level of token probabilities, is narrow the output distribution. Think of it as a contract with a mechanism: the designer defines the valid moves, and the statistical structure of the prompt makes those moves more likely. A contract can be violated intentionally. A probability gradient cannot. When the schema specifies a required numeric dose field and the input contains no dose value, the model does not decide to fabricate — it follows the probability gradient, which points toward the most plausible conforming value.
The schema makes conforming tokens more probable because the schema tokens themselves — field names, brackets, data type specifications — create a context in which value tokens are more probable than non-value tokens. Adding UNCONFIRMED to the schema's controlled vocabulary makes it a conforming token, removing the pressure asymmetry at its source: the model can now satisfy the schema requirement with an honest uncertainty representation rather than a fabricated value. You have not changed what the model knows. You have changed what the model's inference environment makes probable.
Exemplars and schema work at different levels of the inference environment. Exemplars shift which outputs are probable. The schema constrains which outputs are syntactically valid. Both mechanisms are necessary because exemplar influence degrades as inputs diverge from the exemplar distribution, while schema constraints hold regardless of input.
| Architecture | Inference Environment | Failure Mode | Failure Signal |
|---|---|---|---|
| Zero-Shot | Training attractors dominate output distribution | Silent confabulation | None — looks correct |
| Few-Shot + CoT | Exemplars shift distribution; reasoning externalizes working memory | Exemplar bias | Wrong flag — detectable |
| Meta Language DSL | Schema constrains output space to specified tokens | Format overfitting | None — validates cleanly |
The engineering question is not "which one is best?" but "which failure mode can my system tolerate, and how do I make that failure mode visible when it occurs?" A failure mode that produces a validation error downstream is recoverable. A failure mode that produces a confident, well-formatted, clinically plausible record containing a fabricated dose is not.
If your inputs are consistently structured, a Meta Language DSL is appropriate and efficient. If your inputs are heterogeneous — drawn from multiple sources, containing patient self-report alongside clinical notation — then a Meta Language DSL without a carefully designed uncertainty schema is dangerous precisely because it will appear to work. The schema will be satisfied. The records will validate. And some fraction of them will contain values inferred from genre conventions rather than from the actual text.
A practical test: take fifty real inputs from your production environment, not your development set, and examine what fraction contain at least one field that cannot be determined with confidence from the text alone. Fifty inputs is a practical threshold — large enough to sample the tail of your input distribution while remaining feasible to audit manually. In the medication reconciliation case, the engineering team found, on audit, that more than forty percent (illustrative figure) of intake notes contained at least one ambiguous medication entry. Any architecture that handles ambiguity as an exception rather than a first-class state will be wrong forty percent of the time, silently.
In a production pipeline, the extraction layer is not the final consumer of its own output. Its structured records flow into other software systems — pharmacy verification tools, clinical decision support engines, EHR write-back modules — that were built with specific assumptions about what valid input looks like. These are the downstream systems: everything that receives and acts on the extraction output. Their behavior on unexpected input is not a given. It is an engineering decision that must be made explicitly, and it propagates backward to constrain every architectural choice you make in the extraction layer itself.
In a low-stakes extraction task, a silent failure produces a miscategorized ticket or a wrong attribute value. In a clinical extraction task, a silent failure produces a medication record that a pharmacist or physician may act on without further verification. The cost may be an adverse drug event. These failure modes are not on the same scale, and the architecture decision should not be made as though they are.
Uncertainty must be a first-class output state, not an afterthought. Every field in the schema must have an explicitly specified behavior for each of the following epistemic conditions: the value is present and unambiguous; the value is present but conflicting across sources; the value is inferable but not stated; the value is entirely absent; and the value is flagged as uncertain by the source itself. A schema that specifies only the format of a confident, complete extraction has specified only one of five epistemic conditions.
The Meta Language syntax for the medication reconciliation task follows a three-phase structure. The ANALYZE phase is a free-text reasoning trace. The FORMAT phase produces structured records drawn only from ANALYZE findings. The VERIFY phase confirms that nothing was dropped.
ANALYZE: Entry 1 — metformin 500mg, frequency twice daily, patient reports occasional non-adherence to PM dose. Flag: PARTIAL-ADHERENCE. Entry 2 — lisinopril, dose conflicting: family reports 10mg, prescription bottle states 20mg. Flag: CONFLICTING. Entry 3 — aspirin 81mg QD. Confirmed, explicit. Entry 4 — "the little pink one for cholesterol," likely rosuvastatin per PCP note referenced but not attached. Flag: UNCONFIRMED. Entry 5 — warfarin, hold instruction from attending, pre-procedure. Flag: ACTION-REQUIRED. FORMAT: drug_name: [string | UNCONFIRMED] dose: [numeric+unit | CONFLICTING(source_a, source_b) | UNCONFIRMED | null] frequency: [QD | BID | TID | PRN | UNCONFIRMED | null] route: [PO | IV | SQ | UNCONFIRMED | null] status: [ACTIVE | HOLD | DISCONTINUED | UNCONFIRMED] flags: [CONFLICTING | UNCONFIRMED | PARTIAL-ADHERENCE | ACTION-REQUIRED | INFERRED] source: [quote the exact text this record came from] VERIFY: entries_identified_in_analyze: [integer] entries_in_format: [integer] unresolved_flags: [list] reviewer_action_required: [YES | NO] reviewer_action_reason: [string | null]
Before any prompt architecture is deployed on a production input distribution, run it against the hardest twenty percent of your real inputs — the most ambiguous, the most conflicting, the most incomplete — and examine not whether it produces output, but what that output looks like when it is wrong. If wrong outputs are indistinguishable from right outputs, the architecture is not ready for deployment.
The three failure modes described in this chapter share a property that makes them more dangerous than ordinary bugs: their outputs are, by conventional measures, correct. They validate against the schema. They parse without errors. They are indistinguishable, to any automated downstream system, from outputs produced by a well-functioning extraction pipeline. What follows is not a description of how these failures might occur. It is a reconstruction of how they do occur.
Not one of these four entries can be represented as a confident, fully-populated structured record. All four require either null values, uncertainty flags, or conflict markers.
FORMAT: drug_name: furosemide dose: 20mg frequency: QD status: ACTIVE flags: [] source: "water pill, morning" FORMAT: drug_name: metoprolol dose: 25mg frequency: BID status: ACTIVE flags: [] source: "heart medication, cardiologist, post-hospitalization" FORMAT: drug_name: clopidogrel dose: 75mg frequency: QD status: DISCONTINUED flags: [] source: "Plavix, stopped approximately 3 months ago" FORMAT: drug_name: acetaminophen dose: 500mg frequency: PRN status: ACTIVE flags: [] source: "Tylenol, as needed for pain"
Every field is populated. Every record conforms to the schema. No flags are raised. The pharmacy verification system accepted all four records without error. Every dose value is a reasonable inference from clinical genre conventions. None of them is derived from the text.
The FORMAT record looks authoritative for the same reason format overfitting records always look authoritative: it conforms to the schema, it traces its source to the input note, and it contains none of the signals — CONFLICTING markers, UNCONFIRMED tokens, reviewer flags — that would indicate a problem. The only signal of the error is the absence of those tokens. Automated validation will not detect an absence.
The fifth entry — rivaroxaban (Xarelto) 20mg — contains the clinical note that INR is not monitored because rivaroxaban is a direct oral anticoagulant that does not require INR monitoring. This is a clinically relevant piece of information, not a compliance gap.
FORMAT: drug_name: rivaroxaban dose: 20mg frequency: QD status: ACTIVE flags: [PARTIAL-ADHERENCE] ← WRONG source: "Xare1to 20mg w/ evening meal — INR not monitored"
The model interpreted "INR not monitored" as a non-compliance signal, analogous to the metformin non-adherence flag it had seen in a clean exemplar. The model did not learn the PARTIAL-ADHERENCE association from the exemplar alone — it activated a prior from training, built across millions of clinical documents. One exemplar was enough to confirm it. The fix requires a counterexample strong enough to compete with a training-distribution prior.
ANALYZE: Warfarin entry. Hold instruction present. When a hold instruction is present, status is HOLD and dose is recorded as last confirmed dose. Outside records are authoritative over patient self-report when a discrepancy exists.
ANALYZE: Warfarin. Hold instruction present from hematology. Outside records take precedence per exemplar reasoning. Dose: 2.5mg. FORMAT: drug_name: warfarin dose: 2.5mg status: HOLD flags: [ACTION-REQUIRED] source: "Coumadin, dose discrepancy flagged by pharmacy, hold per hematology"
The ACTION-REQUIRED flag is present. The HOLD status is correct. But the dose is 2.5mg rather than CONFLICTING — resolved by a precedence rule that should not have applied when the outside records themselves are under active pharmacy dispute. The error is fully documented in the ANALYZE trace. It is invisible in the FORMAT output. In a production pipeline that discards the reasoning trace, this failure is an archaeological record of a decision that was already operationalized before anyone looked.
This exercise is not a thought experiment. You will make one change to a working extraction pipeline, run it against a specific input, and observe a specific failure. The failure will be silent. The output will look correct. Your job is to see it anyway.
In med_extraction.py, locate the FORMAT schema definition. Change one line:
# Before dose_field = "dose: [numeric+unit | CONFLICTING(source_a, source_b) | UNCONFIRMED | null]" # After — THE ONE LINE CHANGE dose_field = "dose: [numeric+unit]"
1. For the lisinopril entry (conflict: 10mg vs. 20mg), what dose value will appear?
2. For the rosuvastatin entry (no dose stated), what dose value will appear?
3. Will the VERIFY phase catch either failure? State your reasoning.
Signal 1: The lisinopril conflict will appear in ANALYZE and disappear in FORMAT. This is format overfitting in its exact mechanism.
Signal 2: The rosuvastatin dose will contain a numeric value not found anywhere in the input note.
Signal 3: Check whether VERIFY flags the ANALYZE-to-FORMAT discrepancy, or only checks schema compliance. These are different checks.
Signal 4: The rosuvastatin source field will look correct. The dose will be fabricated. Automated validation will not detect the inconsistency.
The VERIFY phase is generated by the same model that generated FORMAT. A model that resolved a conflict silently in FORMAT may evaluate its own output as consistent in VERIFY — because the same reasoning is still in its context window. One candidate fix: pass only the ANALYZE trace and FORMAT output to VERIFY, excluding the task description and exemplar set. Whether this fully escapes the problem, or merely displaces it, is what this chapter leaves open.