Editable video creation based on embedded simulation engine and GAN

Abstract With the progress of artificial intelligence, the embedded generation of images and videos by AI has become a hot topic. This technology is tried to be applied in real-time processing of monitors, cameras and smart phones. Using GAN embedder networks, some research attempts to transfer the style, action and content of one video to another target video. Unfortunately, this generation is often difficult to control. We have created an Engine-GAN (E-GAN) model process, which effectively combines engine method with embedder GAN content to create "real" videos that can be edited in real time. This makes the image and content generated by AI directly controlled. We have made progress in E-GAN architecture, E-GAN workflow, tag generation and entity stylization. We use cuckoo algorithm to optimize the migration target and improve the migration efficiency.

[1]  Deepu Rajan,et al.  Image colorization using similar images , 2012, ACM Multimedia.

[2]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Andrew Owens,et al.  Ambient Sound Provides Supervision for Visual Learning , 2016, ECCV.

[4]  Azlan Mohd Zain,et al.  Cuckoo Search Algorithm for Optimization Problems - A Literature Review , 2013, ICIT 2013.

[5]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[6]  Jitendra Malik,et al.  Learning to See by Moving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Abhinav Gupta,et al.  Unsupervised Learning of Visual Representations Using Videos , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[10]  Fuquan Zhang,et al.  A Video Coloring Method Based on CNN and Feature Point Tracking , 2017 .

[11]  Alexei A. Efros,et al.  Unsupervised Visual Representation Learning by Context Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Stephen Lin,et al.  Intrinsic colorization , 2008, SIGGRAPH 2008.

[13]  Kristen Grauman,et al.  Learning Image Representations Tied to Ego-Motion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Andrew Owens,et al.  Visually Indicated Sounds , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Mohsen Akbari,et al.  Financial forecasting using ANFIS networks with Quantum-behaved Particle Swarm Optimization , 2014, Expert Syst. Appl..