At the latest JP Morgan Healthcare Conference in 2025, NVIDIA and Innophore announced an exciting collaboration to launch an AI-driven drug safety screening platform named CavitOmiX by the end of January 2025. This platform aims to improve the efficiency and accuracy of drug design through the analysis of protein binding sites, marking a revolutionary transformation in the field of drug development. Christian Gruber, CEO of Innophore.

Recently, Nvidia announced the addition of three safety features on its NeMo Guardrails platform, aimed at helping businesses better manage and control AI chatbots. These microservices target common challenges in AI safety and content moderation, offering a range of practical solutions. Among them, the Content Safety service can review content before the AI responds to users, detecting any potential harmful information. This service helps prevent...

Recently, OpenAI showcased its more proactive red team testing strategy in the field of AI safety, surpassing its competitors, especially in the critical areas of multi-step reinforcement learning and external red team testing. The two papers released by the company establish new industry standards for enhancing the quality, reliability, and safety of AI models. The first paper, 'OpenAI's AI Model and System External Red Team Testing Methodology,' highlights the effectiveness of specialized external teams in identifying security vulnerabilities that internal testing may overlook. These external teams consist of cyber...

Mindgard has announced the completion of $8 million in funding aimed at accelerating its research and development efforts and expanding its presence in the U.S. market. The financing round was led by .406Ventures, with participation from Atlantic Bridge, Willowtree Investments, and existing investors IQ Capital and Lakestar. Additionally, Mindgard has appointed two industry leaders to take on the roles of Head of Product and Head of Marketing.

Recently, an incident in a Facebook mushroom enthusiasts group has once again raised concerns about the safety of AI applications. According to 404Media, an AI agent named FungiFriend infiltrated a northeastern mushroom identification and discussion group with 13,000 members and provided potentially fatal erroneous advice. When asked how to cook Sarcosphaera coronaria mushrooms, which contain high levels of arsenic, FungiFriend not only incorrectly stated that the mushroom...

French AI startup Mistral AI has officially launched a new content moderation API aimed at competing with OpenAI and other industry leaders, while addressing the growing concerns around AI safety and content filtering. This service is based on Mistral's Ministral8B model, finely tuned to detect potential harmful content across nine different categories, including pornography, hate speech, violence, dangerous activities, and personally identifiable information.

Recently, the results of an AI weapon scanning pilot project in New York City left many astonished. The scanning equipment provided by a tech company named Evolv failed to detect a single gun but flagged 12 knives instead. The image source note: Images generated by AI, Image licensed by service provider Midjourney. This trial, which started at the end of July, lasted only a month and produced disappointing results. According to CBS New York, although the equipment detected 12 knives, it did not result in any arrests.

In the current rapid development of artificial intelligence technology, Anthropic has recently announced an update to its 'Responsibility Expansion Policy (RSP)', which aims to effectively manage the potential risks posed by high-capability AI systems. As the company behind the popular chatbot Claude, this move by Anthropic clearly aims to find a balance between the continuously enhancing AI capabilities and the necessary safety standards. This new policy introduces what is called capability thresholds, as an additional safety assurance when AI model capabilities are increased.

Recently, California's artificial intelligence safety bill SB1047 has sparked intense competition between Hollywood and Silicon Valley. While it remains unclear whether California Governor Gavin Newsom will sign the bill, several high-profile celebrities have come forward to urge him to support the legislation. The image credit note states: Image generated by AI, image provided by Midjourney. This open letter has over 125 signatures from Hollywood stars, including Ava DuVernay, Jane Fonda, and J.J. Abrams. The letter states, 'We firmly believe that artificial intelligence...

Recently, U.S. officials announced that a global Artificial Intelligence (AI) Safety Summit will be hosted in San Francisco in November. As AI technology advances rapidly, governments around the world are striving to keep pace with this change. The summit will be co-chaired by U.S. Secretary of Commerce Gina Raimondo and Secretary of State Antony Blinken, and is scheduled for November 20-21. Raimondo stated in a release, 'As AI develops rapidly, we are doing everything we can.' She noted the importance of close international collaboration and thoughtful coordination.

Recently, OpenAI's CEO Sam Altman announced his decision to leave the company's internal safety committee, which was established in May of this year to oversee key safety and security decisions in the development and deployment of OpenAI's AI models. Altman's decision has drawn significant attention as he hopes to hand this role over to third-party experts to enhance transparency, collaboration, and security. With Altman's departure, this safety committee will transform into an 'independent oversight committee' focused on safety and security issues.

Recently, the US Department of Commerce released a proposal requiring artificial intelligence (AI) companies to provide detailed reports, aimed at stricter regulation of this rapidly evolving technological field. The notice, titled 'Proposed Rules,' was published on September 9, with the intention of ensuring the safety, reliability of AI technology, and preventing potential misuse by foreign adversaries or non-state actors. Photo credit: Image generated by AI, image licensed from Midjourney. Secretary of Commerce Gina Raimondo stated at a press conference that the rapid development of AI...

The United States, the United Kingdom, and the European Union have jointly signed the first legally binding AI treaty, the 'AI Framework Convention', aimed at ensuring that AI applications adhere to principles of human rights, democracy, and the rule of law. The treaty requires signatory countries to follow principles that protect user data, respect the law, and operate transparently, implementing these principles through legislation and administrative measures. Although compliance with the treaty primarily relies on monitoring, and enforcement may be weak, it is seen as an important blueprint for countries to formulate AI laws. Currently, the US, EU, UK, and several other countries have signed, and the treaty will come into force in five...

Safe Superintelligence (SSI) was founded by former OpenAI Chief Scientist Ilya Sutskever, aiming to develop safe AI systems that surpass human capabilities. The company has raised $1 billion and is valued at $5 billion, focusing on the research and development of AI safety and superintelligent technologies rather than rushing to market. SSI values integrity and potential in its hiring principles, aiming to ensure that AI technology serves humanity and reduces risks through innovative thinking.

Highwire has launched an AI-powered safety risk management tool that provides contractors with deep insights through automated document reviews and risk identification, reducing thousands of man-hours spent on traditional manual reviews. The tool offers more accurate contractor assessments, helping managers implement mitigation strategies before work begins, categorizing safety protocols as general, detailed, or comprehensive, and going beyond simple pass/fail analysis to support more informed decision-making. This AI technology was developed at Harvard University and aims to serve large companies with mature safety programs as well as small contractors, with the goal of enhancing safety.