Forcing Output Formats: From Prompt Tricks to Guaranteed Schemas

There are two very different ways to get clean JSON out of a model: politely asking for it in the prompt, and having the API mechanically enforce it. Only one of them actually guarantees it.

The prompt-level approach

The simplest way to shape output: describe the format explicitly ("respond with exactly these three fields, no extra text") and show one example. This is a prompt pattern - words in, words out, no special API support needed. It works most of the time, but that's the key phrase: a model can still add a stray sentence before the JSON, or slightly misname a field.

The API-level approach

Structured outputs (also called JSON mode, or schemas via tool use) is different: a feature built into the API where you hand over a formal JSON schema, and the response is mechanically constrained to match it - not just asked to, but unable to produce anything that doesn't. Both Anthropic's and OpenAI's platforms offer this for current models.

Pattern vs. feature

A prompt-level instruction is portable (any model, any API) but probabilistic. A schema-based feature isn't portable the same way, but it's deterministic for models that support it - "very likely correctly formatted" versus "guaranteed to validate."

Validation and retry as the fallback

Where a real schema feature isn't available, the standard safety net is: parse the output, validate it against your schema, and if it fails, send it back with the specific error and ask for a fix. Cheap and framework-agnostic, though it costs an extra round trip.

EXAMPLE

Prompt-only (probabilistic): "Respond with only JSON: {\"name\": string, \"age\": number}" API-level (guaranteed), Claude Messages API: { "model": "claude-sonnet-5", "messages": [...], "output_config": { "format": { "type": "json_schema", "schema": {"type": "object", "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}, "required": ["name", "age"]} } } }

🛠️ EXERCISE — TRY IT YOURSELF

Compare prompt-only formatting against a schema-enforced structured output for the same extraction task.

Pick a small extraction task, like pulling a name, date, and amount out of a short paragraph of text.
Write a prompt-only version asking for JSON with those three fields, no schema feature.
Run it 5 times with slightly different input text and check whether the output always parses cleanly as JSON.
If your API access supports it, rewrite the same task using a JSON schema / structured output feature.
Run the schema version 5 times the same way and compare how often each version produces valid, correctly-shaped JSON.

✅ SELF-CHECK

☐ Did the prompt-only version ever produce invalid JSON or an extra field/preamble across your 5 runs?
☐ Did the schema-enforced version fail to parse even once?

QUICK QUIZ

What's the key difference between a prompt-level format instruction and an API-level structured output feature?