REGEN: Learning Compact Video Embedding with (Re-)Generative Decoder

1 Adobe Research    2 Northeastern University

ArXiv Project Gallery

Reconstruction Comparison
4X
Reconstruction Comparison
8X
Reconstruction Comparison
16X

Reconstruction Comparison
32X
Text-to-Video Generation
32X Latent

Reconstruction Comparison (16× Temporal Compression, zdim=8)

We provide visual examples for reconstruction at 16x temporal compression (8x8x16) with 8 latent channels and we compare with MAGVIT-v2 at the resolution of 512x512. With the increasing temporal compression rate, MAGVIT-v2 exhibits much more severe visual artifacts and our method maintains much better spatiotemporal structure than MAGVIT-v2.

Reference

MAGVIT-v2

REGEN