OpenAI's video generation tool Sora has attracted much attention since its launch, but its origins remain a mystery. Now, it seems a piece of the puzzle has been revealed: Sora's training data likely contains a wealth of gaming live streams and tutorial videos from Twitch!
Sora is like a master "mimic," capable of generating videos up to 20 seconds long with just a text prompt or an image, and it can handle various aspect ratios and resolutions. When OpenAI first introduced Sora in February, they hinted that the model had "honed its skills" through videos from Minecraft. So, besides Minecraft, what other gaming secrets might be hidden in Sora's "skill book"?
The results are surprising; Sora seems to be well-versed in various game genres. It can generate a clone game video with a "Mario" theme, despite some minor "flaws"; it can simulate thrilling first-person shooter gameplay, resembling a fusion of Call of Duty and Counter-Strike; it can even recreate fighting scenes from the 90s Teenage Mutant Ninja Turtles arcade game, evoking childhood memories.
Even more astonishing, Sora appears to have a deep understanding of Twitch live streams, suggesting it has "watched" a significant amount of live content. The screenshots of videos generated by Sora not only accurately capture the structure of live streams but also replicate the likeness of famous streamer Auronplay, including the tattoo on his left arm.
Moreover, Sora "recognizes" another Twitch streamer, Pokimane, and has generated a character video resembling her appearance. Of course, to avoid copyright issues, OpenAI has implemented filtering mechanisms to prevent Sora from generating videos featuring trademarked characters.
Although OpenAI has been tight-lipped about the sources of its training data, various signs indicate that gaming content was likely included in Sora's training set. OpenAI's former CTO Mira Murati did not directly deny during a March interview with The Wall Street Journal that Sora used content from YouTube, Instagram, and Facebook for training. OpenAI also acknowledges in Sora's technical specifications that it utilized "publicly available" data and licensed data from media libraries like Shutterstock.
If gaming content was indeed used for Sora's training, it could raise a host of legal issues, especially as OpenAI develops more interactive experiences based on Sora. Intellectual property lawyer Joshua Wagensberg from Pryor Cashman notes that unauthorized use of gaming videos for AI training carries significant risks, as training AI models typically requires replicating the training data, which often contains a wealth of copyrighted content.
Generative AI models like Sora operate on probabilities. They learn patterns from vast amounts of data to make predictions. This capability allows them to "learn" how the world works. However, there are risks; under specific prompts, the model may generate content that closely resembles its training data. This has caused strong discontent among creators who feel their works have been used for training without permission.
Currently, Microsoft and OpenAI are facing lawsuits for allegedly copying licensed code with their AI tools. AI art applications like Midjourney, Runway, and Stability AI are also facing accusations of infringing on artists' rights. Major music companies have filed lawsuits against startup companies Udio and Suno for developing AI song generators.
Many AI companies have long argued for the principle of "fair use," claiming their models create "transformative" works rather than direct copies. However, gaming content is unique. Copyright lawyer Evan Efron from Dorsey & Whitney points out that game videos involve at least two layers of copyright protection: the game content copyright held by game developers and the unique video copyright created by players or video creators. For some games, there may even be a third layer of rights, namely user-generated content copyrights.
For instance, Fortnite allows players to create their own game maps and share them with others. A game video about these maps involves at least three copyright holders: Epic, the gamers, and the map creators. If the court finds that AI model training carries copyright liability, all these copyright holders could potentially become plaintiffs or sources of authorization.
Furthermore, Wagensberg notes that games also contain many "protectable" elements, such as proprietary textures, which judges may consider in intellectual property lawsuits.
Currently, several game studios and publishers, including Epic, Microsoft (which owns Minecraft), Ubisoft, Nintendo, Roblox, and CD Projekt Red (developer of Cyberpunk 2077), have not commented on the matter.
Even if AI companies win these legal disputes, users may not be exempt from liability. If a generative model replicates copyrighted works, those who publish the work or incorporate it into other projects may still face intellectual property infringement charges.
Some AI companies have set up indemnification clauses to address such situations, but exceptions often exist. For example, OpenAI's terms only apply to enterprise clients, not individual users. Additionally, beyond copyright risks, there are also risks of trademark infringement, such as the potential inclusion of assets used for marketing and branding, including characters from games.
As interest in world models grows, the situation could become more complex. One application of world models is generating video games in reality; if these "synthesized" games are too similar to the content used for model training, it could lead to legal issues.
According to Avery Williams, an intellectual property litigation attorney at McKool Smith, training AI platforms in games using elements such as voice, actions, characters, songs, dialogues, and artwork constitutes copyright infringement. The issues of "fair use" raised in numerous lawsuits against generative AI companies will have the same impact on the video game industry as they do on other creative markets.
