Structured Output is a technology that forces LLM outputs to conform to a strict schema such as JSON Schema.

When LLM responses returned as free text are passed directly to business systems, parse failures and unexpected missing fields occur frequently. With Structured Output, you can stably obtain responses in a format that business systems can read directly, simply by specifying a JSON Schema in the response_format of the OpenAI API.

This article is intended for engineers and architects who want to directly connect LLMs to their own systems. It walks through every step required for production use, from JSON Schema design principles to implementation, validation, and error handling in TypeScript, as well as integration patterns with ERPs and asynchronous queues.

Have you ever instructed an LLM to "return the order details in JSON format," only to receive {"item": "coffee"} one day and the sentence "The order is for coffee." the next?

Passing free-text output directly to a business system causes parse failures and unexpected data loss. Even if you specify a format in the prompt, LLMs operate probabilistically and cannot guarantee output consistency. As a result, complex preprocessing and exception handling pile up on the receiving system, driving up integration costs.

Structured Output is a mechanism that eliminates this problem at its root. By forcing the model's output to conform to a schema, the application side can abandon the assumption that "the format might be broken." Only when output format stability is guaranteed does it become practical to design LLMs directly into business systems.

In the early stages of integrating an LLM into a business system, it is tempting to think that "instructing the model to 'return JSON' in the prompt is sufficient." In practice, however, free-text output becomes a serious source of instability in system integration.

With traditional free-text output, parse processing tends to be an unreliable gamble — succeeding if a regular expression can match the content, and failing if it cannot. With Structured Output, the model generates output according to the type constraints of a JSON Schema, which greatly stabilizes parse processing on the application side.

The reason stability improves is that the shape of the output is guaranteed in advance. Types such as string, number, and boolean are enforced at the schema level, making type-cast errors unlikely, and keys specified in required fields are always included in the output. In addition, the depth of arrays and objects is fixed according to the schema, so unexpected structural changes do not occur.

From the perspective of conditional branching, when output fits within a single fixed schema, setting strict: true is effective for keeping parse processing simple. On the other hand, when multiple output patterns are anticipated, it is safer to define the branching on the schema side using oneOf or anyOf and have the receiving code evaluate a type field.

Stabilizing parse processing also changes the impact on downstream business logic.

When attempting to connect an ERP or business system to an LLM, have you ever felt, "Why do I have to write this much parse processing just to receive the LLM's output?"

Structured Output structurally reduces this integration cost. The key points of how it works are the following three:

• Upfront schema agreement: Because the field names, types, and required items returned by the LLM are declared in a JSON Schema, the receiving system can know in advance "which keys will be present."
• Elimination of parse processing: Code that parses free text using regular expressions or string splitting becomes unnecessary, and the output can be converted to an object with a single JSON.parse call.
• Centralized validation: Because the schema definition serves as a single source of truth, the same schema can be referenced across each layer — frontend, backend, and ERP integration.

For example, when extracting order data with an LLM, custom logic was previously required to pull "part number," "quantity," and "delivery date" from text. With Structured Output, simply defining these fields in the schema causes the LLM to return formatted JSON. The ERP side can POST the received JSON directly to an API endpoint, significantly reducing the development cost of intermediate transformation layers.

Conclusion: The success or failure of implementation hinges on three factors: model selection, schema design, and development environment setup. Insufficient preparation upfront leads to rework in later stages.

Before beginning Structured Output implementation, three preparations are required: confirming supported models, establishing a JSON Schema design policy, and setting up dependent libraries. Each step is explained in order.

It is easy to assume that "any latest model can use Structured Outputs," but in practice, support varies depending on the combination of model and version, making prior verification essential.

OpenAI's Structured Outputs (json_schema method) are officially documented as available for GPT-4o, GPT-4 Turbo, and models from gpt-3.5-turbo-1106 onward. The legacy json_object method is deprecated, and the standard policy for new implementations is to use the json_schema method.

The quality of schema design has a significant impact on the model's adherence rate. The simpler and more clearly defined the schema, the more faithfully an LLM tends to follow its instructions.

Core Design Principles

• Limit fields to the essential minimum: Aim for no more than 10 properties per schema. The more fields there are, the higher the likelihood that the model will populate them with incorrect values.
• Specify types concretely: Go beyond string and make use of enum and format (e.g., "format": "date") to explicitly define the range of acceptable values.
• Do not omit description: Adding a description to each field makes it easier for the model to correctly interpret the intended meaning.

Case-by-Case Decisions on Type Definitions

Whether a field is required or optional must be determined based on its use case. Fields used for writing to business systems should be included in required, while supplementary or future-extension fields should be defined as optional using nullable. On the other hand, when you want the LLM to determine whether a value is present, the recommended approach is to allow null while designing with the assumption that post-processing will be handled at the validation layer.

"Which libraries do I need to get up and running as quickly as possible?"—answering this question before starting implementation prevents rework down the line.

For TypeScript, the minimal configuration is as follows:

• openai (official SDK): Natively supports structured output via response_format
• zod: A library for type-safe schema definition and runtime validation
• zod-to-json-schema or the openai SDK's built-in zodResponseFormat utility: Automatically converts Zod schemas to JSON Schema

An example installation command is as follows:

The LTS version of Node.js (18 or higher) is recommended. It includes native fetch support, which stabilizes the SDK's dependencies.

For managing environment variables, use dotenv in conjunction and avoid hardcoding OPENAI_API_KEY directly in your code. In CI/CD pipelines, a configuration that delegates to a secret management service such as GitHub Actions Secrets or Azure Key Vault is recommended.

Conclusion: Implementation can be completed in 3 steps: schema design, API parameter configuration, and validation.

Structured Output implementation proceeds in the following order: schema definition, integration with the API, and response verification. By following each step correctly, you can build a stable pipeline that passes LLM output directly to business systems.

A common stumbling block for many implementers at the first step of schema design is the mindset of "we can always extend it later." In practice, if you don't lock down type strictness and field structure during the initial design phase, downstream system integrations become fragile.

Core Principles for Schema Definition

Start by enumerating all fields required in the output, then assign each an appropriate type.

• Explicitly declare strings as "type": "string" and numbers as "type": "number" or "integer"
• Use "enum" for enumerated values to prevent the model from returning arbitrary strings
• List all required fields in the required array, and intentionally separate optional fields

For example, when retrieving order data, starting with a minimal structure of order_id (string), amount (number), and status (enum: "pending" / "confirmed" / "cancelled") improves model compliance.

Alignment with Prompt Design

Alongside schema definition, it is important to explicitly convey the structural intent in the system prompt as well. An instruction like "Please respond in JSON only" is insufficient. Adding specific language such as "Please output according to the following fields, using the specified types" makes it easier for the model to conform to the schema.

Once the schema definition is complete, the next step is configuring parameters for the API call. In the OpenAI API, Structured Outputs are enabled by passing {"type": "json_schema", "json_schema": {...}} to response_format. Since the legacy "json_object" approach is now deprecated, choose the "json_schema" approach for all new implementations.

The basic configuration in TypeScript is as follows.

The strict flag defaults to false, but it is recommended to set it to true for business system integrations.

When an API returns JSON, "it should match the schema, yet parsing fails for some reason" is a complaint commonly heard in the field. Even with Structured Outputs enabled, incomplete JSON can be returned due to network interruptions or model context overflow, making a validation layer essential.

A robust approach is to implement response handling in the following three stages.

• JSON parsing: Wrap JSON.parse() in a try-catch to catch syntax errors first
• Schema validation: Use a library such as Zod or ajv to verify schema conformance
• Business rule validation: Confirm value ranges and the semantic consistency of required fields

Below is an implementation example using TypeScript + Zod.

Conclusion: Once structured output is obtained, it can be connected directly to systems via three pathways: automatic ingestion into an ERP, an asynchronous queue, and type-safe code generation.

Once you have received schema-conformant JSON, the next challenge is how to feed it into actual business systems. The following sections walk through implementation patterns for each of three scenarios: REST API integration, asynchronous processing, and type sharing with the frontend.

It is tempting to think of automatic ERP ingestion as simply "fire off a POST request and see what happens," but in practice, inserting a Structured Output schema validation step before sending the data can reduce downstream rollback processing to nearly zero.

When passing JSON returned by an LLM directly to an ERP's REST API endpoint, it is important to standardize the following flow.

1. Schema validation: Immediately upon receiving the LLM response, validate it against the predefined JSON Schema. Detect type mismatches and missing required fields at this stage.
2. Field mapping: If the key names in the LLM output do not match the field names expected by the ERP API, convert them in a thin mapper layer. Keeping this layer independent minimizes the blast radius when the schema changes.
3. Idempotency key assignment: Attach an idempotency key (e.g., Idempotency-Key) to the request header to prevent duplicate registrations on the ERP side.
4. Error rollback: If the ERP returns a 4xx / 5xx response, rather than regenerating the LLM output itself, first check the mapping layer logs to isolate whether the issue lies in the field definitions.

A simplified TypeScript implementation example looks like the following.

When submitting large volumes of documents to an LLM in bulk, running synchronous API calls in parallel as-is tends to trigger frequent rate limit errors, making the overall process unstable. Introducing an asynchronous queue allows you to achieve both throughput and reliability.

For small workloads (on the order of a few dozen items), parallel execution with Promise.allSettled is sufficient. However, for batch processing of hundreds of items or more, distributing work via a message queue such as BullMQ or AWS SQS is the appropriate approach.

Have you ever implemented Structured Output on the backend, only to encounter runtime errors caused by a mismatch with the type definitions on the frontend? This problem can be resolved by consolidating the "single source of truth" for types into a JSON Schema.

The basic flow for type generation is as follows:

• Place the JSON Schema in a shared directory in the repository
• Use a tool such as json-schema-to-typescript (json2ts) to automatically generate TypeScript type definitions (.d.ts)
• Both the frontend and backend import and use these generated type definitions

With this approach, any change to the schema automatically updates the type definitions, allowing inconsistencies to be detected at compile time.

Here is a summary of key points in the implementation example:

Because the LLM response is retrieved with strict: true, no fields outside the schema will be included. This means the frontend can avoid writing extra validation code.

Conclusion: Post-implementation failures tend to concentrate in three areas — schema design, null handling, and version management.

Even after adopting Structured Output, schema-related issues are still prone to occur. The H3 sections below outline common failure patterns and how to avoid them.

It is tempting to assume that a more detailed schema will yield higher accuracy, but in practice, the more complex the schema becomes, the more likely the model is to fail to comply with it.

Typical examples of problematic structures include:

• Objects with 4–5 or more levels of nesting
• Definitions that pack 30 or more fields into a single schema
• Complex type branching that makes heavy use of oneOf / anyOf

When given a schema of this complexity, the model may return output that omits required fields or misinterprets types. While OpenAI's Structured Outputs supports a subset of JSON Schema, larger schemas consume more tokens and place greater pressure on the processing capacity allocated to "schema comprehension" within the context.

The mitigation strategy is "splitting and staging."

• Keep the number of fields in any single API call's schema to around 10–15
• Split complex objects across multiple calls and merge the results on the application side
• Use $defs to define reusable sub-schemas, improving overall readability

Additionally, when strict: true is set, all fields in the schema must be listed under required. Leaving optional fields out of required can cause validation errors, so caution is needed (this is covered in detail in the next section).

Handling optional fields may appear straightforward at first glance, but it is an area that easily becomes an implementation pitfall.

A common mistake is failing to account for the fact that when a field is defined in a JSON Schema without being included in the required array, the model may omit that field entirely. If the receiving code accesses response.discount_rate directly, it will encounter an undefined error or null reference exception because the field itself does not exist.

An important distinction to be aware of here is the difference in behavior between a field being omitted and a field being returned as null. When a field is omitted, the key does not exist in the object at all; when it is returned as null, the key exists but its value is empty. Because of this difference, validation logic must be written separately for each case.

A frequently occurring failure pattern is defining an optional field in the schema without making its type nullable — for example, using ["string", "null"]. The moment the model returns null, a validation error occurs. Similarly, code that accesses fields directly without using ?? or ?. for default value handling will cause a runtime error when a key is missing. The mitigation is straightforward: set strict: true, include all fields in required, and explicitly declare "type": ["string", "null"]. This allows both omission and null to be handled intentionally and under control.

"It was working until last week, but suddenly I'm getting parse errors starting today" — this type of inquiry frequently arises in environments where schema version management is not properly established.

When updating a JSON Schema, adding fields tends to preserve backward compatibility, whereas removing fields, changing types, or modifying required flags constitutes a Breaking Change. The moment you modify a prompt or schema definition on the LLM side, there is a risk that existing receiver-side code will break.

Common Breaking Change Patterns

• Changing a field from type string to type number
• Promoting an optional field to required
• Renaming a field (e.g., price → unit_price)
• Changing the structure of a nested object

Recommended Management Practices

1. Manage schema files in a code repository and track changes via Pull Requests
2. Add a version field to the schema so that the receiving side can check the version and branch its processing accordingly
3. When introducing breaking changes, create a new endpoint or new schema ID and allow a migration period during which the old and new versions run in parallel

Q1. Can Structured Output be used with any model without fine-tuning?

It is not available for all models. In the case of OpenAI, Structured Outputs — which specify json_schema in response_format — are only supported on compatible models such as GPT-4o, GPT-4 Turbo series, and gpt-3.5-turbo-1106 or later. Fine-tuning is not required, but support varies depending on the model version, so it is recommended to check the official documentation for the list of supported models before use. For open-source models, whether equivalent functionality is provided varies on a per-model basis.

Q2. How should I decide between strict: true and strict: false?

Setting strict: true causes the model to generate output that strictly conforms to the defined schema. For use cases where parse failures cannot be tolerated — such as automated data ingestion into business systems — strict: true is appropriate. On the other hand, strict: false (the default) allows looser adherence to the schema, making it better suited for flexible exploration during prototyping or testing phases. In production environments, it is generally advisable to select strict: true and thoroughly validate the schema design before moving to operation.

Q3. Can the concept of Structured Output be applied to formats other than JSON (e.g., YAML or XML)?

OpenAI's Structured Outputs feature is designed around JSON Schema, so there is no built-in mechanism to force output directly in YAML or XML. However, it is possible to present a YAML or XML template to the LLM via a prompt and then validate the output with a parser afterward. A more reliable approach is to design a pipeline that first receives structured output in JSON and then converts it to YAML or XML at the application layer, which is superior in terms of stability.

Q4. When changing a schema, how can I minimize the impact on existing integrated systems?

Adding fields tends to preserve backward compatibility, but removing fields or changing types can constitute breaking changes. It is effective to include a $schema version field in the schema and increment the version number upon changes. Additionally, a phased migration approach — running old and new schemas in parallel during a transition period, and retiring the old version only after confirming that downstream systems have adapted to the new version — is recommended for maintaining the stability of system integrations.

Q5. Can the same implementation approach be used with Azure OpenAI?

Azure OpenAI's Structured Outputs supports the same subset of JSON Schema as OpenAI, and the method of specifying response_format is essentially identical. While endpoints and authentication methods are Azure-specific, schema design and validation logic can be reused as-is. However, since the deployment status of supported models and the timing of feature availability may differ by region, it is recommended to check the latest support status in the Azure documentation.

It is easy to assume that "it should work with any model," but in practice, verifying supported models comes first. Structured Outputs (forced JSON Schema output) depend on the model's architecture and training, so they cannot be applied to arbitrary models.

Summary of Support Status

• OpenAI's response_format: { type: "json_schema" } approach is available for GPT-4o, GPT-4 Turbo, and gpt-3.5-turbo-1106 or later models
• The legacy "json_object" approach is deprecated, and migration to the "json_schema" approach is recommended
• Azure OpenAI also supports the same subset of JSON Schema, but the available model versions depend on the deployment configuration on the Azure side

Whether Fine-Tuning Is Required

Fine-tuning is generally not required. Structured Outputs work solely through prompt design and the configuration of the response_format parameter. However, the following points require attention.

To state the conclusion upfront: OpenAI's Structured Outputs is a specification built around JSON, and it is not possible to directly enforce YAML or XML output via a schema.

That said, practical solutions differ depending on the use case.

• When YAML is required: A stable approach is to have the LLM output in JSON and then convert it to YAML on the application side using a library such as js-yaml. Since schema validation is completed at the JSON stage, type safety is preserved.
• When XML is required: Similarly, inserting a JSON → XML conversion step (e.g., using fast-xml-parser) into the pipeline is a practical solution. However, if there are elements that are difficult to express in JSON — such as XML attribute structures — it is necessary to define types at the schema design stage with the post-conversion structure in mind.

It is technically possible to instruct the LLM via a prompt to "output in YAML format," but in this case the JSON Schema enforcement does not apply, which significantly reduces output stability. This approach is not recommended for business system integrations.

A useful decision-making principle is: if the target system requires YAML or XML, receive the output in JSON and convert it; if the reliability of LLM output is the top priority, maintain JSON Schema-enforced output.