August 7, 2024, OpenAI today announced the introduction of a groundbreaking feature in its API—structured output. This new feature ensures that the output generated by the model fully conforms to the JSON schema provided by developers, significantly enhancing the reliability and precision of the API. Unlike previous JSON schemas, structured output not only defines the JSON schema (i.e., the structure of the JSON output) but also guarantees that the JSON output is 100% correct. Additionally, OpenAI announced a price reduction for the new model, with input prices cut by half and output prices reduced by one-third.
Technological Innovation: The Introduction of Structured Output
Last year, OpenAI launched the JSON schema as a powerful tool for developers to build applications. However, while the JSON schema increased the likelihood of generating valid JSON output, it could not guarantee that the output conformed to a specific schema. The introduction of structured output aims to address this limitation by constraining model output to match a predefined pattern, ensuring the accuracy and consistency of the output data.
Performance Improvement: Evaluation and Comparison
In OpenAI's tracking evaluation of complex JSON schemas, the new model gpt-4o-2024-08-06 achieved a perfect score of 100% in structured output, while the old model gpt-4-0613 scored less than 40%. This significant progress not only showcases OpenAI's technological leap but also brings higher work efficiency and lower error rates for developers.
Practical Application: How to Use Structured Output
OpenAI's API has introduced two forms of structured output:
1. Function Call: Structured output can be used by setting it in the function definition. This feature is applicable to all tool-supported models, including all versions and higher. Once structured output is enabled, the model output will match the provided tool definition.
API Request Setup
API Response
2. New Option for Parameters: response_format: Developers can achieve structured responses to user requests by providing the JSON schema json_schema as a new option for parameters.
Security and Support: Secure Structured Output and SDK Updates
Security is always OpenAI's top priority. The new structured output feature adheres to existing security policies, allowing the model to reject unsafe requests when necessary. Additionally, OpenAI's Python and Node SDKs have been updated to natively support structured output, simplifying the development process.
Diverse Use Cases: From Dynamic Interfaces to Data Extraction
The application scope of structured output is extremely broad, including dynamically generating user interfaces based on user intent, separating final answers from supporting reasoning, and extracting structured data from unstructured data. These use cases demonstrate the practicality and flexibility of structured output in different scenarios.
Technical Principles: Constrained Decoding and Alternative Methods
OpenAI achieves 100% matching of model output with JSON Schema through constrained decoding technology. This method dynamically constrains decoding to ensure that the model can only choose valid tokens at each step, thereby reducing the possibility of errors.
Conclusion
OpenAI's structured output feature not only provides developers with a more reliable and precise tool but also further advances the progress of artificial intelligence technology in data processing and application development. With the official rollout of this feature, we look forward to seeing more innovative applications emerge.