Interview with the Founder of Anthropic: Scaling and Reinforcement Learning, Interpretability and AGI Safety

OpenAI's New System Blocks Bio and Chemical Risk Information to Ensure AI Safety

OpenAI recently launched a new system designed to monitor its latest AI reasoning models, o3 and o4-mini, to block prompts related to biological and chemical threats. The system aims to prevent the models from providing suggestions that could incite harmful attacks, ensuring AI safety. OpenAI states that o3 and o4-mini exhibit significantly improved capabilities compared to previous models, thus potentially posing new risks in the hands of malicious users. According to OpenAI's internal benchmarks, o3...

Anthropic Enhances AI Model Safety Measures to Ensure Responsible Scaling

AI company Anthropic recently updated its "responsible scaling" policy, outlining which models require additional safety protections. This move aims to mitigate potential risks before new technologies are released. According to Anthropic's blog, if stress testing reveals an AI model could assist a "resource-constrained state actor" in developing chemical and biological weapons, Anthropic will delay the technology's public launch.

New Regulations for AI Safety and Application; Market Regulation Bureau Accelerates Standard Development

Anthropic Quietly Removes Biden-Era AI Safety Pledge: A Significant Shift in Policy Direction

Former OpenAI Policy Chief Questions Shift in Company's AI Safety Narrative

AI Safety Pledge Evaporates? Anthropic Quietly Removes Biden-Era Partnership; Tech Giant Shift?

Study Finds AI Models Trained on Insecure Code Become Poisonous

Nvidia Launches Three Major AI Safety Tools to Control AI Chatbots

Recently, Nvidia announced the addition of three safety features on its NeMo Guardrails platform, aimed at helping businesses better manage and control AI chatbots. These microservices target common challenges in AI safety and content moderation, offering a range of practical solutions. Among them, the Content Safety service can review content before the AI responds to users, detecting any potential harmful information. This service helps prevent...

Tencent Hunyuan Releases New Theory on Floating Point Quantization Training, Revealing the Limits of Large Model Training

In today's rapid development of Large Language Models (LLM), the training and inference costs of models have increasingly become a focus of research and application. Recently, the Tencent Hunyuan team released an important study that delves into the 'Scaling Laws' of low-bit floating point quantization training, which refers to the principles governing the scale of floating point quantization training. The core of this research lies in exploring how to significantly reduce computational and storage costs without sacrificing performance by lowering the precision of the model.

OpenAI Sets a New Standard in AI Safety: Major Release of Red Team Testing Innovations

Recently, OpenAI showcased its more proactive red team testing strategy in the field of AI safety, surpassing its competitors, especially in the critical areas of multi-step reinforcement learning and external red team testing. The two papers released by the company establish new industry standards for enhancing the quality, reliability, and safety of AI models. The first paper, 'OpenAI's AI Model and System External Red Team Testing Methodology,' highlights the effectiveness of specialized external teams in identifying security vulnerabilities that internal testing may overlook. These external teams consist of cyber...