Text-to-video generation with multi-loss sequential generative adversarial network
Ullah, Anwar, Zhang Xing, Md All Moon Tasir, Mohd Nor Akmal Khalid, Abdul Majid, Naresh Kumar, and Herbert Mukalazi
Scientific Reports · vol. 16 · 2026
Abstract
Generating videos from text remains challenging due to issues like structural complexity, semantic alignment, temporal coherence, and frame discontinuity. It is still highly difficult to generate realistic videos where the frames must comply to both spatial and temporal coherence, despite the fact that Generative Adversarial Networks (GANs) have been successfully deployed to generate videos conditioned on a text description. This study proposes a Sequential Generative Adversarial Network (Seq-GAN) that employs multiple generators and discriminators to generate realistic and high-quality videos using written descriptions, while maintaining semantic alignment and relevance between frames. The approach incorporates a modified Variational Autoencoder (VAE) with reduced and adjusted 2D layers to extract static “gist” features for improved performance. For dynamic frame generation, the model uses three generators ([Formula: see text], [Formula: see text], [Formula: see text]) and three discriminators ([Formula: see text], [Formula: see text], [Formula: see text]) trained with different optimizers: RMSprop, Adam, and SGD. [Formula: see text] and [Formula: see text] generate basic frame structures, while [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] refine video quality and ensure relevance to the text. Experimental results on SBMG, KTH, and UCF-101 datasets show that Seq-GAN achieves notable improvements in Inception Score (IS), Fréchet Video Distance (FVD), and CLIP-similarity confirming the method’s effectiveness in generating realistic, semantically accurate video sequences.
Interactive demo
This interactive is a faithful re-design of the research, simplified for illustrative and teaching purposes. It conveys the idea and behaviour of the model — it is not the paper’s full method, data, or results.
A looping preview of agentic procedural content generation: a genetic loop with generator and critic agents evolves a rough draft into a well-formed role-playing quest, matching a target difficulty arc.
Ullah, Anwar, Zhang Xing, Md All Moon Tasir, Mohd Nor Akmal Khalid, Abdul Majid, Naresh Kumar, and Herbert Mukalazi. “Text-to-video generation with multi-loss sequential generative adversarial network.” Scientific Reports 16 (2026). https://doi.org/10.1038/s41598-026-45313-7
@article{ullah2026texttovideo,
author = {Ullah, Anwar, Zhang Xing, Md All Moon Tasir, Mohd Nor Akmal Khalid, Abdul Majid, Naresh Kumar, and Herbert Mukalazi},
title = {Text-to-video generation with multi-loss sequential generative adversarial network},
journal = {Scientific Reports},
volume = {16},
year = {2026},
doi = {10.1038/s41598-026-45313-7},
}