August 7, 2024, OpenAI today announced the introduction of a groundbreaking feature in its API—structured output. This new feature ensures that the output generated by the model fully conforms to the JSON schema provided by developers, significantly enhancing the reliability and precision of the API. Unlike previous JSON schemas, structured output not only defines the JSON schema (i.e., the structure of the JSON output) but also guarantees that the JSON output is 100% correct. Additionally, OpenAI announced a price reduction for the new model, with input prices cut by half and output prices reduced by one-third.

image.png

Technological Innovation: The Introduction of Structured Output

Last year, OpenAI launched the JSON schema as a powerful tool for developers to build applications. However, while the JSON schema increased the likelihood of generating valid JSON output, it could not guarantee that the output conformed to a specific schema. The introduction of structured output aims to address this limitation by constraining model output to match a predefined pattern, ensuring the accuracy and consistency of the output data.

Performance Improvement: Evaluation and Comparison

In OpenAI's tracking evaluation of complex JSON schemas, the new model gpt-4o-2024-08-06 achieved a perfect score of 100% in structured output, while the old model gpt-4-0613 scored less than 40%. This significant progress not only showcases OpenAI's technological leap but also brings higher work efficiency and lower error rates for developers.

image.png

Practical Application: How to Use Structured Output

OpenAI's API has introduced two forms of structured output:

1. Function Call: Structured output can be used by setting it in the function definition. This feature is applicable to all tool-supported models, including all versions and higher. Once structured output is enabled, the model output will match the provided tool definition.

carbon.png

API Request Setup

carbon (1).png

API Response

2. New Option for Parameters: response_format: Developers can achieve structured responses to user requests by providing the JSON schema json_schema as a new option for parameters.

carbon (2).png

carbon (3).png

Security and Support: Secure Structured Output and SDK Updates

Security is always OpenAI's top priority. The new structured output feature adheres to existing security policies, allowing the model to reject unsafe requests when necessary. Additionally, OpenAI's Python and Node SDKs have been updated to natively support structured output, simplifying the development process.

carbon (4).png

Diverse Use Cases: From Dynamic Interfaces to Data Extraction

The application scope of structured output is extremely broad, including dynamically generating user interfaces based on user intent, separating final answers from supporting reasoning, and extracting structured data from unstructured data. These use cases demonstrate the practicality and flexibility of structured output in different scenarios.

Technical Principles: Constrained Decoding and Alternative Methods

OpenAI achieves 100% matching of model output with JSON Schema through constrained decoding technology. This method dynamically constrains decoding to ensure that the model can only choose valid tokens at each step, thereby reducing the possibility of errors.

Conclusion

OpenAI's structured output feature not only provides developers with a more reliable and precise tool but also further advances the progress of artificial intelligence technology in data processing and application development. With the official rollout of this feature, we look forward to seeing more innovative applications emerge.