Recently, U.S. officials announced that a global Artificial Intelligence (AI) Safety Summit will be hosted in San Francisco in November. As AI technology advances rapidly, governments around the world are striving to keep pace with this change. The summit will be co-chaired by U.S. Secretary of Commerce Gina Raimondo and Secretary of State Antony Blinken, and is scheduled for November 20-21. Raimondo stated in a release, 'As AI develops rapidly, we are doing everything we can.' She noted the importance of close international collaboration and thoughtful coordination.

Recently, OpenAI's CEO Sam Altman announced his decision to leave the company's internal safety committee, which was established in May of this year to oversee key safety and security decisions in the development and deployment of OpenAI's AI models. Altman's decision has drawn significant attention as he hopes to hand this role over to third-party experts to enhance transparency, collaboration, and security. With Altman's departure, this safety committee will transform into an 'independent oversight committee' focused on safety and security issues.

Recently, the US Department of Commerce released a proposal requiring artificial intelligence (AI) companies to provide detailed reports, aimed at stricter regulation of this rapidly evolving technological field. The notice, titled 'Proposed Rules,' was published on September 9, with the intention of ensuring the safety, reliability of AI technology, and preventing potential misuse by foreign adversaries or non-state actors. Photo credit: Image generated by AI, image licensed from Midjourney. Secretary of Commerce Gina Raimondo stated at a press conference that the rapid development of AI...

The United States, the United Kingdom, and the European Union have jointly signed the first legally binding AI treaty, the 'AI Framework Convention', aimed at ensuring that AI applications adhere to principles of human rights, democracy, and the rule of law. The treaty requires signatory countries to follow principles that protect user data, respect the law, and operate transparently, implementing these principles through legislation and administrative measures. Although compliance with the treaty primarily relies on monitoring, and enforcement may be weak, it is seen as an important blueprint for countries to formulate AI laws. Currently, the US, EU, UK, and several other countries have signed, and the treaty will come into force in five...

Safe Superintelligence (SSI) was founded by former OpenAI Chief Scientist Ilya Sutskever, aiming to develop safe AI systems that surpass human capabilities. The company has raised $1 billion and is valued at $5 billion, focusing on the research and development of AI safety and superintelligent technologies rather than rushing to market. SSI values integrity and potential in its hiring principles, aiming to ensure that AI technology serves humanity and reduces risks through innovative thinking.

Highwire has launched an AI-powered safety risk management tool that provides contractors with deep insights through automated document reviews and risk identification, reducing thousands of man-hours spent on traditional manual reviews. The tool offers more accurate contractor assessments, helping managers implement mitigation strategies before work begins, categorizing safety protocols as general, detailed, or comprehensive, and going beyond simple pass/fail analysis to support more informed decision-making. This AI technology was developed at Harvard University and aims to serve large companies with mature safety programs as well as small contractors, with the goal of enhancing safety.

The 'Self-Regulatory Initiative for the Generative Artificial Intelligence Industry' was launched at the 2024 China Internet Civilization Conference in Chengdu, initiated by the Artificial Intelligence Safety Governance Committee of the China Cybersecurity Association in collaboration with industry, academia, and research institutions. The initiative focuses on five core areas: compliance and safety of data and algorithm models to ensure legal and secure data usage; content ecosystem development to encourage positive content generation and maintain a healthy online environment; technological innovation and quality enhancement to advance technology and quality assessment; ethical standards that respect human rights and social values; and communication, cooperation, and openness.

Changes at OpenAI have sparked broad concern in the industry regarding AI safety issues. The AGI safety team has significantly reduced in size, from about 30 members to around 16, raising worries about whether the company is neglecting AI safety concerns. The departures were not an organized effort, but rather a gradual loss of confidence among team members. Former governance researcher at OpenAI, Daniel Kokotajlo, noted that the reduction in the team appears to be related to the company's increased focus on product and commercialization development. In response to external doubts, OpenAI emphasizes its commitment to safety.

California lawmakers have passed the SB1047 bill aimed at ensuring AI safety, which is set to be signed by Governor Gavin Newsom. The bill requires AI models with training costs exceeding $100 million to undergo safety testing before release and establishes an emergency stop function in case of accidents. It also mandates reporting to the California Attorney General within 72 hours of an incident and establishes a 'Cutting-edge Model Bureau' to oversee compliance, with substantial fines for violators. This initiative aims to prevent AI misuse, ensure safety, and has garnered support from notable figures in the AI field. However, tech giants have had mixed reactions.

Google has announced the reactivation of the character image generation feature for its AI chatbot Gemini, aimed at producing diverse content including photo-realistic landscapes and textured paintings. This feature is powered by Google's AI text-to-image generator Imagen3 and currently supports only English. It will be available to Gemini Advanced, Business, and Enterprise users in the coming days. The newly upgraded Imagen3 model offers improved performance and includes built-in safety measures optimized to avoid historical inaccuracies.

Musk publicly supports California's SB1047 bill, which aims to require large AI model manufacturers to implement safeguards against potential harms. The bill mandates documentation of strategies to prevent serious damage caused by AI. Musk underscores the necessity of AI regulation, consistent with his past concerns about AI risks. His AI company, xAI, will be bound by the SB1047 bill. In contrast, OpenAI opposes the SB1047 bill and supports an alternative proposal. This event highlights the complexity and controversy of AI regulation within the industry.

Musk supports California's SB1047 bill, which requires large AI model development companies to document safety measures to prevent models from causing serious harm. He emphasized that he has always advocated for AI regulation, believing it should be treated like the regulation of risky products and technologies. Musk's company xAI will be bound by this law, even though he plans to leave California. In contrast, OpenAI openly opposes SB1047 and supports an alternative bill. Adobe and Microsoft, on the other hand, support another new California bill AB3211, which requires labeling of AI-generated content. These developments highlight the dynamic landscape of AI regulation.

OpenAI and several tech companies oppose a California bill aimed at ensuring the safety and transparency of artificial intelligence, fearing that it will hinder innovation and development in AI technology, potentially leading to an outflow of the industry. OpenAI stated in a press release that overly strict regulations could stifle the creativity of developers and researchers, limiting exploration and experimentation. They emphasized that technological innovation requires flexibility rather than being constrained by legal limitations. OpenAI hopes to engage in dialogue with lawmakers to promote a more reasonable legal framework that aligns with industry development trends. If the bill passes, it could lead to...

Anthropic has announced an expansion of its vulnerability reward program aimed at testing a 'next-generation AI safety mitigation system,' primarily focusing on identifying and defending against 'universal jailbreak attacks.' Special attention is given to high-risk areas, including CBRN defense and cybersecurity. Participants will have the opportunity to engage with the new safety system ahead of time, identifying vulnerabilities or bypassing security measures, with rewards of up to $15,000. This initiative aims to enhance the safety of AI systems by attracting security researchers to collaboratively discover and fix potential threats, setting a benchmark for safety in the AI industry.

OpenAI's report documents in detail the advantages and risks associated with the GPT-4o model. The model exhibits peculiar behaviors under certain conditions, such as mimicking user voices in high background noise environments, which may stem from the model's difficulties in understanding speech. GPT-4o occasionally generates disturbing non-verbal sounds or effects under specific prompts, although there is evidence showing the model usually declines such requests. Furthermore, the report indicates that GPT-4o might inadvertently involve music copyright issues without authorization. To address these risks, OpenAI has...

Gary Marcus criticizes Sam Altman, the CEO of OpenAI, in his column, pointing out the discrepancies between Altman's statements on AI safety and his actual actions, raising questions about integrity. Marcus expresses concerns about the safety risks of generative AI like ChatGPT, criticizing Altman's overly optimistic attitude as disconnected from reality, and believes that the safety of AI technology may never be guaranteed. He emphasizes the lack of sufficient regulation and transparency in AI research and development, posing risks to users and society.

OpenAI CEO Sam Altman announced that OpenAI will provide early access to its new generative AI model for safety testing by the U.S. AI Safety Institute. This decision aims to counter accusations of neglecting safety in AI development, following criticism for disbanding a team focused on AI safety measures. In response to concerns, OpenAI pledged to allocate 20% more computing resources to safety research and removed non-disparagement clauses for ....

Google has launched a series of online safety measures aimed at eliminating deepfake content from search results, particularly explicit images without consent. When users request removal of such content, the system not only processes the specific item but also filters out related results and even removes duplicates. To prevent such content from appearing prominently in search results, Google has adjusted its ranking algorithm to prioritize high-q....

Recently, OpenAI supported three key AI legislations in Congress aimed at influencing the federal government's policy on artificial intelligence. These bills include the 'Future AI Innovation Act', which seeks to establish a U.S. AI Safety Institute as a federal agency for developing standards and guidelines for AI models; the 'National Science Foundation AI Education Act', which provides federal scholarships for AI research and establishes AI educational resources within the education system; and the 'Create AI Act', designed to minimize risks associated with new technologies. OpenAI's move not only demonstrates its commitment to...