With the proliferation of deepfake technology, the amount of false content online has surged. According to data from the ID verification platform Sumsub, the number of deepfakes quadrupled from 2023 to 2024, accounting for 7% of all fraud in 2024. This includes various forms such as identity impersonation, account takeover, and complex social engineering attacks. To tackle this challenge, Meta has recently launched a new watermark tool - Meta Video Seal, aimed at helping with AI-generated content.

AI technology is like a double-edged sword; it brings convenience but also poses risks. To better manage these risks, California Governor Gavin Newsom recently signed nine bills specifically regulating AI-generated content, particularly focusing on the phenomenon of deepfakes. Note: The image is AI-generated, and the image licensing provider Midjourney is referenced. Among these bills, SB1047 stands out as it requires AI developers to implement safety measures to mitigate risks.

Recently, California Governor Gavin Newsom signed eight laws addressing challenges in the field of artificial intelligence. These laws aim to tackle issues related to the existential risks posed by AI systems, the deepfake production of nude images by AI generators, and the creation of AI clones of deceased actors by Hollywood studios. Among these, SB926 criminalizes the creation and distribution of deepfake nude images and defines using AI-generated nude images resembling others for blackmail as illegal. The SB981 bill requires social media platforms to establish user reporting channels for relevant content.

During a recent signing ceremony, California Governor Gavin Newsom announced a series of new laws aimed at the artificial intelligence (AI) industry, which are relatively strict on a national level. The new regulations comprise five items, three of which specifically address AI deepfake technology that could impact elections, while the other two concern the cloning of Hollywood actors' voices and images. Image source note: Image generated by AI, image authorized by service provider Midjourney. The new law AB2655 requires large online platforms...

With the continuous advancement of artificial intelligence technology, the difficulty of forging images and videos has decreased significantly, making the deepfake phenomenon increasingly severe. Identifying these false contents has become an urgent issue to address. Recently, a research team from Binghamton University conducted an in-depth discussion on this topic, using frequency domain analysis techniques to reveal the characteristics of AI-generated images, thus helping people identify false information. Image source note: The image is AI-generated, with permission from image licensing provider Midjourney. This research was conducted by the Department of Electrical and Computer Engineering.

Recently, Google has taken action to combat non-consensual deepfake content in response to public concern and legislative pressure. Following the Taylor Swift incident eight months ago, tech companies and lawmakers began seriously addressing this issue. Google announced measures to prevent pornographic deepfake content from appearing in search results, including streamlining the process for victims to remove non-consensual pornography, filtering pornographic search results, removing duplicate images, and optimizing search results to reduce the exposure of deepfake content. Henry Ajdah has expressed support for Google's initiatives, believing it will significantly reduce the exposure of non-consensual content.

As one of the fastest-growing forms of adversarial artificial intelligence, the losses related to Deepfakes are expected to skyrocket from $12.3 billion in 2023 to $40 billion by 2027, with a compound annual growth rate of an astonishing 32%. Deloitte predicts a surge in deepfakes in the coming years, with banks and financial services becoming primary targets. Deepfakes represent the cutting edge of adversarial AI attacks, with a growth rate of 3,000% just last year. It is projected that Deepfak

Google DeepMind has conducted research on the most common malicious applications of AI for the first time, revealing that AI-generated 'deepfakes' of false politicians and celebrities are more prevalent than attempts to use AI-assisted cyber attacks. This study was a collaboration between Google's AI division DeepMind and its R&D unit Jigsaw, aimed at quantifying the risks of AI generation tools that have been pushed to market by the world's largest tech companies seeking massive profits.

Google will participate in the Global Partnership on AI (GPAI) summit, committing to make machine learning and artificial intelligence more beneficial for users. The company will take steps to address synthetic media, including labeling modified or synthesized content on Search and YouTube. Google has invested $1 million to establish a multidisciplinary Responsible AI Center at the Indian Institute of Technology Madras. Google is dedicated to the development of responsible artificial intelligence and is taking action to combat the deepfake challenge.

Microsoft has developed an AI tool that can generate new avatars and voices or replicate a user's appearance and speech. The tool, named Azure AI Speech, has been trained on human images and can create realistic avatars using AI. Users can upload videos to replicate the voice and appearance of specific individuals. Microsoft has emphasized the tool's limitations and protective measures to prevent misuse. Critics are concerned that this tool could be used to create deepfake content.

Microsoft has released a digital watermark tool named Content Credentials as a Service, aimed at protecting political campaigns from the threats posed by deepfakes. The tool appends metadata including content creation time, method, creator, and more. Microsoft is collaborating with organizations to launch a reliable election information website on Bing and supports legislation banning AI-generated fake political ads. In addition to the digital watermark tool, Microsoft also offers cybersecurity protection and AI recommendations, establishing an election communication center.

CBS News has established a verification department focused on addressing AI deepfakes and misinformation. The department is led by CBS News and television's Senior Vice Presidents Claudia Milne and Ross Dagen. CBS plans to hire specialists, invest in new technologies, and produce segments explaining misinformation to help viewers better understand it.

👾 Deepfake detection startup Reality Defender has successfully raised $15 million, led by DCVC. 🛡️ Reality Defender offers various tools to help businesses, government agencies, and others identify and combat synthetic media. 🌐 In addition to the funding, the company has released explainable AI features and real-time deepfake screening tools.

🤖 Issues with new AI features on social platforms: New AI features rolled out by Meta and Snapchat have already encountered various problems, including deepfakes and children's data privacy. 🔎 Regulatory attention: The Federal Trade Commission convened a roundtable to discuss copyright and other related issues. Regulators are scrutinizing the impact of generative AI on various industries and consumers. 💡 Media company responses: BBC has established AI principles, while some publishers are concerned that ChatGPT is siphoning traffic. Some publishers discussed this at the Digiday Publishing Summit.