Recently, Adobe announced a brand-new creative tool — Adobe Firefly Video Model. This innovative tool marks Adobe's bold venture into the realm of AI-generated videos, built upon its existing Firefly generative AI image model.
Adobe states that this model is ethically trained, using data that the company owns or has permission to use, ensuring the safety of the generated content.
However, some creators on Adobe Stock have expressed dissent, feeling that their uploaded works were not intended to be used for training AI models, and even potentially competing with their own styles.
Adobe Firefly Video Model is expected to enter the testing phase later this year, and the company has already opened a waiting list for early access, which interested individuals can apply for.
In terms of functionality, it supports not only text-to-video and image-to-video conversion but also video editing capabilities, though each generated clip is limited to a maximum of 5 seconds.
High-Quality Video Generation Examples
Adobe's official blog has showcased some early and impressive examples of generated videos, with remarkable content quality and a high degree of adherence to text prompts.
Using Firefly text-to-video, one can generate seamless B-Roll to fill gaps in the timeline with text prompts, various camera controls, and reference images.
For instance, inputting a scene description like "close-up of a reindeer in a snowy forest at sunset, with dreamy and soft lighting" yields an excellent video effect.
Specific prompt: "Cinematic closeup and detailed portrait of a reindeer in a snowy forest at sunset. The lighting is cinematic and gorgeous and soft and sun-kissed, with golden backlight and dreamy bokeh and lens flares. The color grade is cinematic and magical."
Similarly, inputting a scene like "slow-motion volcanic eruption, with the camera flying over lava" also results in captivating footage.
Specific prompt: "Slow-motion fiery volcanic landscape, with lava spewing out of craters. The camera flies through the lava and lava splatters onto the lens. The lighting is cinematic and moody. The color grade is cinematic, dramatic, and high-contrast."
According to official introductions, the Firefly Video Model excels in generating videos of natural landscapes. When producers miss key establishing shots for setting scenes, the model can generate inserts with camera movements, such as landscapes, plants, or animals.
Prompt: Footage of a camera on a drone flying over a desert with wind blowing over the dunes creating waves in the sand below.
Additionally, if more supplementary shots are needed, clips based on reference frames can be generated to fill gaps in the timeline.
For example, the original video might look like this:
Supplementary video can be generated using the Firefly Video Model:
Prompt: Detailed extremely macro closeup view of a white dandelion viewed through a large red magnifying glass
Furthermore, one can also use a variety of camera controls (such as angles, movements, and zooms) to create perfect perspectives on the generated videos.
For business decision-makers, Adobe's Firefly video model could be a highly attractive option, especially for companies looking to produce videos for employee training or marketing.
Adobe also provides legal assurances, offering protection against infringement lawsuits even when using its AI models, although it is not yet clear whether this includes the Firefly video model.
Since its launch in March 2023, Adobe has integrated the Firefly model into its widely-used Creative Cloud software suite as the cornerstone of new AI features.
Firefly tools have been embedded into popular applications like Photoshop, Illustrator, and Lightroom, empowering users with functionalities such as Generative Fill, Generative Remove, and Text-to-Template.
According to Adobe, since its launch, over 1.2 billion images and vectors have been generated using Firefly, making it one of the company's fastest-adopted innovations.
Ashley Still, the Adobe executive leading the project, stated that the company "is working closely with the video editing community to advance the Firefly Video Model."
Not Just Supporting AI Video Generation
Additionally, the Firefly video model is not just about generating new videos; it also includes AI editing features such as removing unwanted elements and perfect transitions.
With the rise of short video content and compressed production times, Adobe hopes these tools will help editors speed up their workflows while maintaining high-quality output.
Adobe's Firefly Video Model provides tools that help editors not only cut footage but also manage color correction, animation, visual effects, and more.
The model also supports a wide range of creative effects, including the generation of b-roll, macro shots, and atmospheric elements (such as fire, smoke, or water), which users can overlay on pre-recorded or animated footage.
A standout feature coming to Premiere Pro later this year is Generative Extend. This feature will allow editors to extend video clips, filling gaps in the footage or smoothing transitions — helping to match the clip's rhythm with audio and other visual elements.
Adobe demonstrated how Generative Extend can extend the duration of a shot to perfectly align with the climax of music:
Adobe is also exploring how to integrate AI into more creative workflows, hoping that these new tools will help creators have more time to explore creativity, thereby enhancing the overall quality of projects.
Official blog: https://blog.adobe.com/en/publish/2024/09/11/bringing-gen-ai-to-video-adobe-firefly-video-model-coming-soon
Key Points:
🌟 Adobe's Firefly Video Model supports text-to-video, image-to-video, and video editing functionalities, expected to enter the testing phase later this year.
🎥 Early generated videos exhibit astonishing quality, showcasing a high degree of adherence to text prompts, attracting significant attention from creators.
📈 Adobe provides legal assurances to help businesses mitigate infringement risks when using AI to produce videos.
