In the digital era, content security issues have increasingly garnered attention. On September 26th, OpenAI officially launched a new multimodal content moderation model named "omni-moderation-latest".
This model is based on the latest GPT-4o technology, capable of accurately identifying and detecting harmful text and images. This update will provide developers with powerful tools, enabling them to build more robust moderation systems.
The highlight of the new model is that it supports moderation of both text and image inputs, especially excelling in handling non-English content.
Compared to previous moderation models, "omni-moderation-latest" not only improves the accuracy of identification but also enhances the ability to detect a wider range of harmful content. It can evaluate categories such as violence, self-harm, and sexual content, ensuring users can communicate in a safer environment.
Since launching the Content Moderation API in 2022, the volume and variety of content that automated moderation systems need to handle have been increasing, especially as more AI applications enter large-scale production. Today, many companies, such as Grammarly and ElevenLabs, are using this API to protect user safety and prevent the creation of inappropriate content.
The advantages of the new model are reflected in several aspects:
Firstly, it can perform multimodal harmful content classification, evaluating combinations of images and text to identify risks related to violence and sexuality.
Secondly, the model has added two new categories of text moderation, specifically related to illegal and violent content, further enhancing its moderation capabilities.
Additionally, the accuracy of detecting non-English content has significantly improved, with tests showing a 42% increase in accuracy across 40 languages, particularly outstanding in low-resource languages.
For developers, this new moderation model remains accessible as a free Content Moderation API. OpenAI hopes this upgrade will allow more users to leverage the latest research and security systems, creating a more friendly online experience for users.
Official blog: https://openai.com/index/upgrading-the-moderation-api-with-our-new-multimodal-moderation-model/
Key Points:
📊 The new model "omni-moderation-latest" is based on GPT-4o technology, supporting multimodal moderation of text and images.
🌍 Detection accuracy has improved by 42% for 40 languages, particularly outstanding in low-resource languages.
🔒 Added two new categories of text moderation, enhancing the ability to identify illegal and violent content.