Imagine being able to see a person speaking, moving, and even performing, all from just a single photo within seconds. This is the allure of OmniHuman-1, recently launched by ByteDance. This artificial intelligence model, which has gone viral online, can create highly realistic videos that bring static images to life. Combined with audio clips, it achieves lip synchronization, full-body movements, and rich facial expressions. Unlike traditional deepfake technologies, OmniHuman-1 is not limited to facial replacement; it can animate the entire person.

With the rapid development of artificial intelligence technology, deepfake content has gradually attracted widespread social attention. To address this emerging issue, California has introduced a series of new laws and regulations at the beginning of the new year, aimed at strengthening the regulation of deepfake content generated by artificial intelligence. These laws will affect social media platforms, content creators, and related industries, marking the entry of AI technology application regulation into a new phase. Note on image source: Image generated by AI, image licensed by Midjourney. One of the important laws expands the scope.

Recently, at the 12th Digital Finance and Technology Finance Conference, part of the Zhongguancun Forum series of events, Ant Group's Deepfake detection solution was selected as a “Financial Technology Innovation and Application Case.” Leveraging its Tiantong Laboratory, Ant Group has constructed the first large-scale, high-quality, multi-modal Deepfake dataset in the industry, synthesizing over a million multimedia contents that fully simulate Deepfake attack samples in real-world financial risk control environments, becoming a benchmark for evaluating existing Deepfake detection in the financial sector.

Meta has recently launched a new neural watermark system called VideoSeal, designed to help identify AI-generated videos, even if the videos have been edited, it can effectively trace their origin. This watermark is invisible to viewers but can be verified through detection methods to confirm the video's authenticity. Meta has released the entire system as open source software under the MIT license, including the VideoSeal watermark model, research papers, and training and inference code.

Recently, a video has sparked widespread attention online, featuring an account impersonating Zhang Wenhong in a live stream selling a type of protein bar. Many netizens began to question, 'Is Zhang Wenhong really selling products?' Even more concerning is that some elderly viewers believed the video to be genuine, with some placing orders and sharing the information in social groups. After investigation, it was confirmed that the Zhang Wenhong in the video was not a real person, but a deepfake generated through AI technology. Zhang Wenhong has responded, indicating that there is more than one such fraudulent account.

Today, Deepfake is ubiquitous. With the rise of generative AI, the amount of false content online has exploded. According to identity verification platform Sumsub, the number of Deepfakes globally has increased fourfold from 2023 to 2024. By 2024, Deepfake will account for 7% of all fraud incidents, involving everything from identity impersonation and account theft to complex social engineering attacks. To effectively combat Deepfake, Meta...

With the proliferation of deepfake technology, the amount of false content online has surged. According to data from the ID verification platform Sumsub, the number of deepfakes quadrupled from 2023 to 2024, accounting for 7% of all fraud in 2024. This includes various forms such as identity impersonation, account takeover, and complex social engineering attacks. To tackle this challenge, Meta has recently launched a new watermark tool - Meta Video Seal, aimed at helping with AI-generated content.

Recently, a federal lawsuit regarding a Minnesota law on 'using deepfake technology to influence elections' has sparked new controversy. The plaintiff's legal team pointed out in the latest legal documents that the affidavits supporting this law may contain text generated by artificial intelligence. Image source note: The image was generated by AI, provided by Midjourney according to reports from the Minnesota Reformer, where the state's Attorney General Keith Ellison had requested the Stanford Social Media Lab's assistance.

After the 2024 U.S. presidential election, the rapid spread of misinformation online severely impacts Americans' perceptions of candidates, as well as public health, climate change, and immigration topics. The development of generative AI, along with its ability to create deepfake content and its tendency to 'hallucinate' facts, may further exacerbate this issue. Image credit: Image generated by AI, licensed from Midjourney. To address this challenge, Norwegian startup Factiverse was born.

Singapore has recently announced a series of important measures in cybersecurity, particularly regarding safety guidelines for Artificial Intelligence (AI) systems and legislation banning the use of deepfake technology in election advertising. These initiatives aim to help organizations mitigate potential risks during the development and deployment of AI, ensuring the security of the technology. Image Credit: The image is generated by AI and licensed through Midjourney. The 'AI System Security Guidelines' launched by the Cyber Security Agency of Singapore (CSA) emphasize the concept of 'security by design.'

AI technology is like a double-edged sword; it brings convenience but also poses risks. To better manage these risks, California Governor Gavin Newsom recently signed nine bills specifically regulating AI-generated content, particularly focusing on the phenomenon of deepfakes. Note: The image is AI-generated, and the image licensing provider Midjourney is referenced. Among these bills, SB1047 stands out as it requires AI developers to implement safety measures to mitigate risks.

Recently, California Governor Gavin Newsom signed eight laws addressing challenges in the field of artificial intelligence. These laws aim to tackle issues related to the existential risks posed by AI systems, the deepfake production of nude images by AI generators, and the creation of AI clones of deceased actors by Hollywood studios. Among these, SB926 criminalizes the creation and distribution of deepfake nude images and defines using AI-generated nude images resembling others for blackmail as illegal. The SB981 bill requires social media platforms to establish user reporting channels for relevant content.

During a recent signing ceremony, California Governor Gavin Newsom announced a series of new laws aimed at the artificial intelligence (AI) industry, which are relatively strict on a national level. The new regulations comprise five items, three of which specifically address AI deepfake technology that could impact elections, while the other two concern the cloning of Hollywood actors' voices and images. Image source note: Image generated by AI, image authorized by service provider Midjourney. The new law AB2655 requires large online platforms...

Recently, several major American artificial intelligence companies made a solemn pledge to the White House, stating that they will prevent their AI products from being used to generate non-consensual deepfake pornography and child sexual abuse materials. Six companies, including Adobe, Anthropic, Cohere, Microsoft, OpenAI, and the open-source web data repository Common Crawl, signed this non-binding commitment to ensure that their products are not misused to create harmful images of gender violence.

In today's world, deepfakes and synthetic media have become increasingly common due to the advancements in generative artificial intelligence (gen AI). To tackle this challenge, many organizations are working to improve digital watermarking technologies for tracking the origins of content. While no organization has completely succeeded yet, the Coalition for Content Provenance and Authenticity (C2PA) is expanding its membership. C2PA has announced that Amazon has become a new member of the organization's advisory board. C2PA is an open technical standard organization under the Linux Foundation.

With the continuous advancement of artificial intelligence technology, the difficulty of forging images and videos has decreased significantly, making the deepfake phenomenon increasingly severe. Identifying these false contents has become an urgent issue to address. Recently, a research team from Binghamton University conducted an in-depth discussion on this topic, using frequency domain analysis techniques to reveal the characteristics of AI-generated images, thus helping people identify false information. Image source note: The image is AI-generated, with permission from image licensing provider Midjourney. This research was conducted by the Department of Electrical and Computer Engineering.

Singapore has recently introduced a new bill aimed at banning the use of deepfake technology during elections to ensure the authenticity of information and the fairness of the electoral process. With the advancement of generative AI technology, the risk of deepfake content has increased. The bill seeks to regulate the production of false content created by both AI and non-AI technologies, including videos, audio, and images. The core provisions prohibit the publication of online ads that falsely depict candidates' words and actions during election periods, with violators facing fines or imprisonment. The bill also allows for issuing correction orders to individuals and social media platforms that publish false content.

Microsoft collaborates with the anti-revenge pornography organization StopNCII to provide victims with tools to erase deepfake pornographic images from search engines. By creating a digital fingerprint (hash value) of the images on the device, it helps to completely remove related synthetic nude images from platforms like Bing. This initiative aims to address the issue of deepfake pornography caused by the misuse of generative AI technology, tackling the challenges that cannot be effectively managed solely through user reports. However, Google has not yet partnered with StopNCII, and there is a lack of specific federal laws addressing such behavior in the United States.

As the 2024 U.S. presidential election approaches, generative artificial intelligence (AI) is increasingly becoming a new player on the electoral battlefield. AI technology can generate false images, videos, and audio, influencing voter decision-making, a trend that has manifested in multiple instances. For example, former President Trump shared AI-generated images attempting to mislead the public, while residents of New Hampshire encountered deepfake calls aimed at dissuading them from participating in the Democratic primaries. Experts warn that the impact of AI-generated misinformation could substantially alter election outcomes, and current detection technologies remain underdeveloped.