Alert button
Picture for Shengyuan Huang

Shengyuan Huang

Alert button

Deep Video Harmonization with Color Mapping Consistency

May 02, 2022
Xinyuan Lu, Shengyuan Huang, Li Niu, Wenyan Cong, Liqing Zhang

Figure 1 for Deep Video Harmonization with Color Mapping Consistency
Figure 2 for Deep Video Harmonization with Color Mapping Consistency
Figure 3 for Deep Video Harmonization with Color Mapping Consistency
Figure 4 for Deep Video Harmonization with Color Mapping Consistency

Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background. So far, video harmonization has only received limited attention and there is no public dataset for video harmonization. In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos. Moreover, we consider the temporal consistency in video harmonization task. Unlike previous works which establish the spatial correspondence, we design a novel framework based on the assumption of color mapping consistency, which leverages the color mapping of neighboring frames to refine the current frame. Extensive experiments on our HYouTube dataset prove the effectiveness of our proposed framework. Our dataset and code are available at https://github.com/bcmi/Video-Harmonization-Dataset-HYouTube.

Viaarxiv icon

HYouTube: Video Harmonization Dataset

Sep 18, 2021
Xinyuan Lu, Shengyuan Huang, Li Niu, Wenyan Cong, Liqing Zhang

Figure 1 for HYouTube: Video Harmonization Dataset
Figure 2 for HYouTube: Video Harmonization Dataset
Figure 3 for HYouTube: Video Harmonization Dataset

Video composition aims to generate a composite video by combining the foreground of one video with the background of another video, but the inserted foreground may be incompatible with the background in terms of color and illumination. Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background. So far, video harmonization has only received limited attention and there is no public dataset for video harmonization. In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos. Considering the domain gap between real composite videos and synthetic composite videos, we additionally create 100 real composite videos via copy-and-paste. Datasets are available at https://github.com/bcmi/Video-Harmonization-Dataset-HYouTube.

Viaarxiv icon