Zhou Hongyi discusses the challenges posed by AI hallucinations and their implications for cybersecurity. He emphasizes the need for a balanced approach, neither downplaying nor overstating the risks involved.

In today's rapidly developing artificial intelligence landscape, the phenomenon of AI 'hallucinations' has become increasingly frequent, causing significant inconvenience for many companies. Customer service chatbots confidently describe non-existent products, financial AIs fabricate market data, and medical bots provide dangerous medical advice. These issues are no longer mere curiosities; they pose serious risks that are affecting company reputation and profitability. To tackle this challenge, San Francisco-based startup Patronus AI has announced the launch of the world's first self-service platform aimed at real-time detection and prevention.

Microsoft's trajectory in the field of artificial intelligence is undergoing a significant transformation. Once ambitiously investing in self-developed large models, it now shifts its focus to collaboration with OpenAI and has launched a new 'Correction' feature aimed at addressing the 'hallucination' issues in AI-generated content. From self-development to leveraging partnerships: the evolution of Microsoft's AI strategy. Initially, Microsoft invested a substantial amount of resources in the field of generative AI (GenAI), hoping to create AI models comparable to or even surpassing OpenAI's GPT series. However, during the lengthy development process...

Recently, Google announced the launch of a new open-source AI model, DataGemma, aimed at addressing the 'hallucination' problem that large language models (LLMs) often encounter when handling statistical data. This hallucination phenomenon causes the model to potentially provide inaccurate answers to questions related to numbers and statistics. The release of the DataGemma model marks a significant advancement for Google in the field of AI. Image source note: Image generated by AI, image licensed by service provider Midjourney to reduce hallucinations during statistical queries.

Cornell University and other institutions conducted hallucination benchmarking tests on generative AI models like GPT-4o, Claude, and Gemini, finding that these models produce hallucination-free text only about 35% of the time, indicating that the reliability of AI still has room for improvement. The research involved fact-checking through authoritative sources across topics like law, health, and history, designing question sets that included content not covered by Wikipedia. The results showed that OpenAI's models performed the best overall, but the improvement compared to GPT-3.5 was limited. The study pointed out that model size is not related to the frequency of hallucinations.

Two papers reveal that content generated by generative AI may be regarded as 'nonsense', highlighting the challenges posed to public servants due to its disregard for accuracy. The erroneous information produced by generative AI differs from intentional lies, and the papers argue that using the term 'nonsense' is more precise, aiding the public's understanding of technology. Currently, AI laws and regulations are incomplete and cannot effectively prevent the harm caused by such misinformation. The authors suggest the introduction of regulations similar to those in the publishing field, emphasizing the avoidance of 'casual statements' that could cause social harm, ensuring that truth is not compromised.

While large language models like ChatGPT, Claude, and Gemini are impressive, they all share a significant issue: they often generate hallucinations. This is a serious problem in the field of artificial intelligence, and even Apple has expressed concerns about how future Apple Intelligence will handle hallucinations. Fortunately, a team of researchers has now developed an AI hallucination detector that can determine whether the AI is fabricating content.

Fujitsu has announced the launch of two new AI trust technologies aimed at enhancing the reliability of conversational AI models, including technologies for detecting hallucinations and adversarial attacks. These technologies will be integrated into Fujitsu's AI platform 'Kozuchi', designed to provide tools for businesses and individual users to assess the reliability of conversational AI responses, thereby enhancing AI security. The first technology aims to detect hallucinations in conversational AI responses with high precision, addressing potential misinformation issues in those replies. The second technology is used to detect conversational...