A BLOGGER nicknamed “AI Crazy House” from Sichuan Province released an AI-generated animated short film of “Journey to the West” on the internet. The film shocked netizens with its visual effects and opened up new possibilities for presenting classic works in film and television.

Through deep learning and innovative techniques, AI reinterpreted this Chinese literary masterpiece, delivering breathtaking visual effects and emotional experiences.

The blogger, whose surname is Feng, has 15 years of art experience. He stated that the video, which would have taken at least six months to produce manually, was completed in a week with the help of AI. He also explained the steps to generate videos with AI. Taking the first episode of “Journey to the West” as an example, from the beginning of chaos to the birth of the stone monkey to the learning of art, he emphasized the need to determine the required number of pictures and storyboards.

He mentioned using ChatGPT to analyze the original text, complete storyboard planning, utilize AI painting software for drawing, and then employ “picture-to-video” software to animate the images.

These processes generally produce a large number of preliminary drafts. “At present, it is difficult for AI to fully understand my meaning, and the production is relatively random. Every time, I will use AI to generate a lot of materials to find the appropriate ones for the next step,” Feng said.

Regarding the inclusion of elaborate fight scenes in the “Journey to the West” TV series and other works, Feng noted that there are still many restrictions in AI-generated videos, making such feats difficult to achieve.

OpenAI introduced Sora as their text-to-video model several days up, which can generate videos up to one minute long while maintaining visual quality and adhering to the user’s prompt. It quickly became a global sensation for its enormous potential to revolutionize movie trailers and the short video industry.

Sora is capable of generating complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt but also how those things exist in the physical world.

The current model has weaknesses. It may confuse spatial details of a prompt, for example, mixing up left and right, and may struggle with precise descriptions of events that take place over time, like following a specific camera trajectory.

Liu Xingliang, director of the Beijing-based Data Center of China Internet, a research institute specializing in the internet industry, said Sora is undoubtedly a major breakthrough for AI.

“It not only demonstrates AI’s advanced ability to understand and create complex visual content, but also brings about unprecedented opportunities and challenges for content creation, entertainment and film and television production industries. The video-generation model will help video content creators to turn their ideas into reality at a faster speed and at a lower cost, and offer audience members richer and more diverse visual experiences,” Liu said.

The use of text-to-video AI models raises concerns about ethics, copyright protection, personal privacy leakage and data security.

To ensure the authenticity and transparency of the content has become an important issue, and more efforts are needed to formulate rules and regulations to ensure the healthy development of such technology.

“We’ll be taking several important safety steps ahead of making Sora available in OpenAI’s products. We are working with red teamers  — d omain experts in areas like misinformation, hateful content, and bias  —  who will be adversarially testing the model,” said OpenAI.

Stephanie Laporte, chief executive and founder of the OTTA advertising and influencer agency, believes the technology will “force the industry to evolve.” She also anticipates ad companies with lean budgets will resort to AI tools to save money on workers.

(China Daily)