Adobe is expanding its AI offerings in video, including generative AI tools for Premiere Pro as well as creating a video model for its Firefly model collection.
Adobe Premiere Pro's generative AI can remove items from a scene
Adobe has been increasing its use of AI in its creative tool suite, providing artists and designers with ways to use the fast-developing technology for images. Now, Adobe is turning its attention more towards video.
In a preview of its AI changes on Monday, Adobe claims it will be bringing generative AI elements to Adobe Premiere. The changes are intended to improve video creation and production workflows, helping editors and producers speed up production and come up with new ideas.
Arriving later in 2024, the new generative AI tools will perform actions such as adding and removing objects from a scene, or extending an existing clip.
Generative Extend adds frames to clips to make them longer in length, which gives editors more opportunities to make perfectly timed edits and to add smooth transitions.
Object Addition and Removal lets users select and track objects in a scene and have them replaced by the tool. This can remove unwanted items from view, change elements of a subject's wardrobe, or add things to an otherwise empty scene.
This can help eliminate distracting elements from a scene that takes away from the story or realism of what is being filmed. For example, a period drama may want to get rid of modern items visible in a skyline, such as electrical wires or flying planes.
More traditional generative AI is also present in Text to Video, which creates new footage based on text prompts and reference images. Adobe reckons this can be used to create storyboards, as well as creating B-roll to augment live-action footage.
An example of Adobe Premiere Pro's AI video generation by text prompt.
These changes will be driven by a new video model that will be joining its Firefly AI model range, which already covers imaging, audio, and 3D.
"Adobe is reimagining every step of video creation and production workflow to give creators new power and flexibility to realize their vision," said Ashley Still, Senior Vice President, Creative Product Group at Adobe. "By bringing generative AI innovations deep into core Premiere Pro workflows, we are solving real pain points that video editors experience every day, while giving them more space to focus on their craft."
Last year, Adobe updated Premiere Pro with Sensei-powered text-based editing, as well as changes introducing automatic tone mapping and collaborative editing improvements.
Audio additions
At the same time as its video updates, Apple added that there are AI-based audio changes that will be made generally available to customers from May.
The audio improvements include Interactive Fade Handles for custom audio transitions, and a redesigned waveform that automatically resizes as the track height changes, making it easier to read.
A new Essential Sound badge with audio category tagging will use AI to tag audio clips as dialogue, music, sound effects, or ambiance, with a new icon added to provide one-click access to related tools.
Effect Badges are new visual indicators for audio clips to show which have effects, to let users quickly add new ones, and to automatically open effect parameters from within the sequence.
These join the Enhance Speech tool, an AI-powered effect to remove noise and improve dialogue, which has been available since February.