In recent days, Sora OpenAI has sparked increasing interest due to its ability to create high-quality, realistic videos from text.
However, beyond generating videos from textual descriptions or prompts, Sora from OpenAI can also transform images into video. It is also capable of extending the duration of a video.
This advancement represents a milestone in the AI field, as it allows users not only to devise content from scratch but also to reimagine and expand existing material. The applications of this technology range from animating still images to extending video clips, including the creation of perfect loops, opening up a range of possibilities for creatives and professionals in the sector.
Regarding this, OpenAI’s technical report on Sora details the following:
1. Instructions with images and videos
All the previous results and on our homepage show examples of text-to-video conversion. But Sora can also be asked for other inputs, such as pre-existing images or videos. This capability allows Sora to perform a wide range of image and video editing tasks: creating perfect loop videos, animating still images, extending videos forward or backward in time, etc.
2. Animating images from DALL·E
Sora is capable of generating videos provided an image and a message as input. Below we show example videos generated from images of DALL·E 231 and DALL·E 330.
Imagen de Dall-E y video de Sora @OpenAI pic.twitter.com/q56KXSQduS
— Laura Islas (La chica del SEO) (@ella_laquesefue) February 18, 2024
Imagen generada por DALL-E y animada como video con Sora pic.twitter.com/iMYFkNdytF
— Laura Islas (La chica del SEO) (@ella_laquesefue) February 18, 2024
3. Expanding generated videos
Sora is also capable of extending videos, either forward or backward in time. Below are four videos that were extended backward in time from a segment of a generated video. As a result, each of the four videos starts differently from the others, but all four videos lead to the same ending.
Sora también es capaz de extender videos, ya sea hacia adelante o hacia atrás en el tiempo pic.twitter.com/SrVxfQYzY8
— Laura Islas (La chica del SEO) (@ella_laquesefue) February 18, 2024
4. Video blending
We can also use Sora to gradually interpolate between two input videos, creating seamless transitions between videos with completely different themes and scene compositions. In the following examples, the center videos are interpolated between the corresponding videos on the left and right.
Sora sirve también para interpolar gradualmente entre dos videos de entrada pic.twitter.com/t0MEyFooyU
— Laura Islas (La chica del SEO) (@ella_laquesefue) February 18, 2024
5. Videos with 3D consistency
Sora can generate videos with dynamic camera movement. As the camera pans and rotates, people and elements in the scene move consistently through three-dimensional space.
[SOCIEDAD] OpenAI presentó "Sora", la inteligencia artificial que convierte texto en video: puede crear piezas de hasta 60 segundos de altísima calidad, con movimientos cinematográficos y múltiples personajes. ?pic.twitter.com/9g3NWZaQRY
— ElCanciller.com (@elcancillercom) February 16, 2024
Text to Video
Sora is not the first artificial intelligence model capable of generating video from text. However, there are factors that highlight its breakthrough.
Its ability to interpret textual instructions and convert them into complex scenes, complete with emotionally expressive characters and precise environmental details, is undoubtedly impressive. Users can choose between photorealistic or animated styles, suggesting a wide range of applications, from creating educational content to entertainment production.
However, what makes Sora particularly unsettling is its ability to generate videos that are indistinguishable from reality.
The videos shared by OpenAI generate scenes that never happened, with characters that never existed. Nothing you see here is real, but it appears to be.
Sora OpenAI and the Ethical Dilemma
The use of a diffusion model to smooth videos from static noise to impressive clarity is a technical feat, but it also raises questions about the potential for abuse. Deepfakes, videos manipulated to make it appear as if someone is saying or doing something they never did, are already a significant concern. With technologies like Sora, the fear is that such fakes could become even more convincing and difficult to detect.
OpenAI is aware of these concerns and has noted that Sora is still in development, with limited access to a small group of researchers and creatives for testing and feedback.
The company has acknowledged that, although Sora surpasses competitors like Midjourney and Stable Diffusion in creating longer and more fluid videos, there are still areas that require improvement, especially in understanding cause and effect and spatial awareness.
Expectations around Sora are high. Its ability to generate customized educational content, detailed historical recreations, or visualizations of future products are just some of the potential applications.
ALSO. When will OpenAI Sora be available to the public?
