Search AI Products and News

In today's rapidly developing artificial intelligence landscape, the phenomenon of AI 'hallucinations' has become increasingly frequent, causing significant inconvenience for many companies. Customer service chatbots confidently describe non-existent products, financial AIs fabricate market data, and medical bots provide dangerous medical advice. These issues are no longer mere curiosities; they pose serious risks that are affecting company reputation and profitability. To tackle this challenge, San Francisco-based startup Patronus AI has announced the launch of the world's first self-service platform aimed at real-time detection and prevention.

["Patronus AI Releases Copyright Detection Tool", "OpenAI's GPT-4 is Accused of Serious Infringement Issues", "Microsoft Engineers Warn that AI Image Generation Tools Could Pose Societal Risks"]

Patronus AI has released the SimpleSafetyTests suite, revealing critical security vulnerabilities in AI systems such as ChatGPT. The tests cover five high-priority harm areas, including suicide, child abuse, and bodily harm, exposing severe weaknesses in 11 LLMs. It emphasizes that secure system prompts can reduce unsafe responses, but results indicate that production systems may require additional safeguards. The testing results suggest that LLMs need rigorous and customized safety solutions before handling real-world applications.

Patronus AI is a startup based in the United States that focuses on AI model safety and governance. The company has just completed a $3 million Series A funding round led by Lightspeed China. Patronus AI primarily provides AI model security assessment, monitoring, and enhancement platforms for enterprise clients. The company utilizes adversarial generative networks to offer security assessment services for AI models to its clients. Patronus AI stated that this funding round will accelerate product development and expand its R&D team.