How to get structured JSON output from an LLM

By the end of this guide you will have code that sends text, or a document, to a language model and gets back data that matches a schema you defined: the right fields, the right types, and a clear failure when the model gets it wrong. This is what LLM structured output with JSON means in practice, and it is what makes model output safe to feed into databases, APIs and other code.

There are three broad ways to get there: libraries that validate and retry, libraries that constrain generation as it happens, and services that do extraction for you. This guide shows the steps and compares six tools from our Developer platforms category.

Key takeaways

• Define the shape of the data first (a schema), then choose how to enforce it: validate and retry after generation, or constrain generation while it happens.
• The main decision is library or service. Pydantic AI, Instructor, LMQL and Guidance are code libraries, while Reducto and MyMagic AI are services that extract data for you.
• Schema-valid does not mean correct. A model can return perfectly formatted JSON with the wrong values, so check the content too.
• Constraint support varies by model and backend. LMQL lists partial constraint support on OpenAI Chat API models, so check what your provider supports.

What you need before you start

• A task and a sample input. For example, extracting invoice fields from text or classifying support tickets.
• A schema in mind. A list of fields with types and which are optional.
• A model and credentials. Most libraries here work with several providers. You pay your model provider for calls.
• A programming environment. Pydantic AI and Instructor are Python libraries (Instructor also lists ports for TypeScript, Go, Ruby, Elixir and Rust).
• Time. A working extraction takes under an hour. Hardening it with tests takes longer.
• Difficulty. Moderate; some Python is expected.

Which kind of AI tool fits this job

Validate and retry with typed models. Instructor takes a Pydantic model as the response type, converts it to a schema, checks the model's reply against your rules and retries with the error when validation fails. It supports OpenAI, Anthropic, Google, Mistral, Cohere, Ollama, DeepSeek and more than 15 other providers. Pydantic AI is a fuller agent framework from the Pydantic team where each run returns validated, typed results, with tools, MCP support and durable execution on top.

Constrain generation as it happens. Guidance lets you enforce a regular expression, a fixed list of options, a context-free grammar, or a JSON schema or Pydantic model while the model generates. LMQL is a query language where a where clause sets hard constraints such as length, allowed values, a regex or a type like int, and typed output can produce schema-safe JSON. These work best with backends that support the needed level of control, such as local models.

Document extraction services. Reducto parses PDFs, images and spreadsheets, and its Extract API returns JSON that follows a schema you define. This suits invoices, forms and reports where layout matters.

Batch extraction. MyMagic AI runs a model over many files in your own cloud storage, accepts an optional JSON schema for structured output, and returns results after asynchronous processing.

Step 1: Define the schema

Start with the output, not the prompt.

1. List every field you need, with a name, a type and whether it is required.
2. Use narrow types where you can: enumerations for categories, numbers for amounts, dates for dates.
3. Decide what the model should do when a value is missing. A nullable field is better than inviting a guess.
4. Add short descriptions to ambiguous fields, since the model reads them as instructions.

Check it worked by hand-filling the schema for three sample inputs and confirming it fits each one.

Step 2: Choose an enforcement method

Pick how strictly the structure must hold.

1. For API models and quick extraction, validate and retry with Instructor, or get validated, typed results from Pydantic AI.
2. For local models where you need hard guarantees, consider constrained generation with Guidance or LMQL.
3. For PDFs and scans, look at a document service such as Reducto before building your own parsing.
4. For large batches in cloud storage, consider MyMagic AI.
5. Check provider fit. Guidance's listing says full grammar constraints need a backend with full support for it.

Check it worked by writing one sentence that names the tool, the model and why they fit.

Step 3: Write the extraction call

Get one input working end to end.

1. Define the schema in code. With Instructor and Pydantic AI, this is a Pydantic model.
2. Pass the schema as the expected response type, together with your input text and a short instruction.
3. Keep the instruction specific: what to extract, what to do when data is missing, and what to ignore.
4. Call the model and print the typed result.

Check it worked by seeing a typed object come back rather than a string of text.

Step 4: Add validation rules and retries

The schema checks the shape. Rules check the sense.

1. Add field rules for ranges, formats and allowed values.
2. Add cross-field checks, such as a total that must equal the sum of its lines.
3. Let the library retry on failure. Instructor retries with the validation error, which gives the model a chance to correct itself.
4. Cap the number of retries so a bad input cannot loop forever or run up cost.

Check it worked by feeding a deliberately bad input and confirming the call either corrects itself or fails with a clear error.

Step 5: Handle documents and long inputs

Real inputs are messy.

1. For PDFs and scans, convert them to text or Markdown first, or use a service that extracts to a schema. Reducto lists layout-aware parsing, tables, forms and extraction citations.
2. Split long inputs into sections if they exceed the model's context window.
3. For large batches, use a batch service or your own queue. MyMagic AI returns a job ID that you poll until the job completes.
4. Keep the source reference, such as page or filename, with each extracted record so you can trace errors.

Check it worked by running ten real files and counting how many validate on the first try.

Step 6: Stream, test and monitor

Make the extraction dependable over time.

1. Build a test set of real inputs with correct answers, and run it whenever you change the prompt, schema or model.
2. Measure accuracy per field, not only whether the JSON is valid.
3. Use streaming where it helps the user interface. Instructor lists streaming of partial objects and lists.
4. Log failures and review them. Pydantic AI connects to Pydantic Logfire or any OpenTelemetry backend for observability.

Check it worked by changing the model name and confirming your test set still passes, or at least showing exactly which fields regressed.

Step 7: Review sensitive outputs and ship

Structured does not mean trustworthy.

1. Have a person check extracted values that drive money, legal, medical or compliance decisions.
2. Treat model output as untrusted input: validate it again before writing to a database, and never run it as code.
3. Decide what happens on failure: retry, queue for human review, or reject.
4. Review data handling for personal data sent to the model or to a document service. Reducto lists SOC 2 Type II and HIPAA support with a signed BAA, but you should confirm terms for your use.

Check it worked by sending a failure through your fallback path and seeing it land in the review queue.

Tools for this job compared

Tool	Good fit for	Pricing	Stated limits
Pydantic AI	Typed Python agents with validated outputs, tools and durable execution	Open source	Requires programming; you need credentials per model provider; Logfire is a separate paid service
Instructor	Schema-first extraction with Pydantic models and automatic retries	Open source	Requires programming; each provider supports different modes and capabilities
LMQL	Hard constraints on output with a query language	Open source	Constraint support is partial on OpenAI Chat API models; you need to learn the query syntax
Guidance	Regex, grammar and JSON schema constraints in Python	Open source	Full grammar constraints need a fully supporting backend; has a learning curve
Reducto	Schema-based extraction from PDFs, scans and spreadsheets	Free trial, paid from $10, usage-based (checked 24 Oct 2026)	Dense extractions over 100 fields per page may add a per-field surcharge; Deep Extract is not eligible for the batch discount
MyMagic AI	Batch extraction over files in S3 or Google Cloud Storage	Pricing unknown	Needs an IAM role for your S3 bucket; results arrive after asynchronous processing

Prices and plan limits change, so confirm the current terms on each vendor's site before you build around them.

Common mistakes to avoid

• Trusting valid JSON. A reply can match the schema and still contain wrong values. Fix it by testing accuracy per field against known answers, and by adding rules that catch impossible values.
• Making every field required. Forcing a value invites the model to invent one. Fix it by allowing null or an explicit "unknown" for fields that may be absent.
• Retrying without a limit. Endless retries cost money and hide a bad schema or prompt. Fix it by capping retries and logging each failure reason.
• Assuming every provider behaves the same. Instructor's listing notes each provider supports different modes and capabilities. Fix it by testing your schema on each model you plan to use.
• Passing messy documents straight to the model. Tables and scans lose structure in plain text. Fix it by parsing documents first or using a service built for them, then checking the extracted numbers by hand.

How we put this together

This is an evidence-based guide built from each tool's listing in our directory, which is compiled from the vendor's own site and pricing page. It is not a review based on using the tools. We don't rank tools, and nobody paid to be included. Some outbound links may be affiliate links; see our disclosure.

Frequently asked questions

Is there a free way to get structured JSON from an LLM?

Yes. Pydantic AI, Instructor, LMQL and Guidance are open-source libraries, and Instructor's listing says you only pay your model provider. You can also use a local model through Ollama with Instructor to avoid API charges. Reducto's Standard tier includes $150 in free usage.

What is the difference between validating output and constraining it?

Validation checks the reply after the model has written it and retries if it fails. Constraining limits what the model can generate token by token, so the output follows a grammar or schema from the start. Constraints give stronger guarantees but need backend support, while validation works with any model.

Can an LLM return JSON accurately?

It can return well-formed JSON reliably when you use a schema and validation, but the values can still be wrong. Accuracy depends on the model, the input quality and how clear your field descriptions are. Test against known examples and have a person review outputs that matter.

Which tool is good for extracting data from PDFs?

Reducto parses PDFs, images and spreadsheets and has an Extract API that returns JSON following your schema. Instructor also lists multimodal input, including PDFs, if you prefer a library. Check the vendor's listing for file types, limits and data handling.

Can I keep my data private?

Libraries run in your own code, but your text still goes to whichever model provider you call. Using a local model through Ollama avoids sending text to a third party, and Reducto lists VPC or on-premises deployment for Enterprise customers. Read each provider's data terms before sending sensitive content.

Compare more options in our Developer platforms category.