Adobe Firefly muscles into AI video–here’s what it looks like | usagoldmines.com

Adobe said today that it’s bringing AI-generated video, aka the Firefly Video Model, to Adobe Premiere Pro plus its Firefly generative art service. Unlike its generative AI image capabilities, however, it won’t be free.

AI-generated video has been available for months. In December, OpenAI released Sora, its ability to craft AI video clips of several seconds from a text prompt. What Adobe is offering is authenticity. Like the AI-generated images in Firefly and Photoshop, Adobe trained its Firefly Video Model on licensed clips.

Right now, Adobe is positioning its AI-generated video clips as b-roll. It’s like filler video behind a title, for example. It also helps facilitate a transition between two different scenes.

In a demonstration provided to reporters, Adobe envisioned an AI-generated clip of red-hot sparks flying across the screen, as a backdrop to a title screen introducing clips of Iceland. However, Adobe also showed off an entirely AI-generated “drone flight” over an erupting volcano for the same project, suggesting viewers will still have to question the authenticity of what they see.

Adobe Firefly models will be able to generate and download the video. In Premiere Pro, like in Photoshop, users will be able to generate the AI-generated content within the app themselves and then simply drop it inside the workflow.

Adobe

How much will Adobe Firefly Video cost?

To use Adobe’s Firefly Video Model, however, you’ll need to pay. Adobe is offering early access on two plans. The $9.99 Firefly Standard plan gives users 2,000 video credits per month or up to 20 five-second 1080p videos. Firefly Pro allows users to create 70 five-second 1090p videos via 7,000 video/audio credits per month as well. A Firefly Premium plan for those “looking to generate video content on a regular basis” is also en route, Adobe says, but the pricing hasn’t been announced.

The plans sound a little confusing at first, but essentially you’re using the credits to “buy” the videos, according to an Adobe representative who I asked to clarify the plans. In addition to simply creating the video, users will be able to edit videos, too: by providing filters, crafting paths for the camera to create motion shots, and more. Creatives can also lock the first and last images of the clip to facilitate transitions, and tweak the color to maintain consistency with the other clips in the scene, Adobe says. It doesn’t sound like those features will cost credits.

Adobe

Adobe won’t generate audio via AI, however. Instead, Adobe is offering to take pre-recorded audio and translate it into 20 languages. Adobe is promising that its translations will maintain an “authentic voice,” with voice, tone, cadence, and acoustic matching.

AI-generated video is far more computationally intensive than AI-generated still images, so it makes sense that Adobe would charge more. Adobe already uses a credit system for AI images, and Firefly only allocates 25 image-generation credits to users who sign up for the free version of Adobe’s apps. With the video model, Adobe has yet to allow users to trial the Firefly Video model with a free tier.

What isn’t clear is if users will “spend” any available credit, or if they’ll just be used to buy video creation time.

Adobe

Adobe does have competition here. Creating AI-generated video via cloud services is possible with services like Sora, Pika Labs, Runway, and more. Services like Invideo.ai also offer to help create entire video segments from scratch, using AI-generated clips (as well as stock images) and an AI-generated script.

Creating AI video on your own local PC is also possible — barely, it seems. Mochi, according to the documentation, requires 60GB of VRAM to generate 480p videos from your PC. That’s more than the 32GB of VRAM in an Nvidia GeForce 5090 Founder’s Edition. Other competitive offerings don’t seem to be much better.

Local AI video models seem to struggle with creating a video clip from a text prompt rather than a reference image, too. Adobe’s Firefly model does both. Running an AI generator like Stable Diffusion on your PC seems quite realistic for AI-generated still images, but AI video may be forced to use the cloud for this generation of PCs.

Adobe’s offering that sort of power with its cloud service. In the long run, convenience may be the real selling point. While Adobe’s Firefly service provides a grab-and-go approach to AI art, it’s the easy, tight integration into Photoshop that’s made it especially useful. I’d expect the same of Premiere Pro and Adobe’s new video model.