Label-Free Object Detection in Videos of Physical Interactions with GANs

Object recognition is currently one of the most important problems in computer vision. Most approaches to object recognition focus on supervised learning methods, which often need large amounts of labeled data in order to train. In this paper, we pursue an unsupervised learning approach to object recognition by incorporating a physics prior. We attempt to train a Generative Adversarial Network on two tasks: detecting the ball in a Pong game, and detecting the trajectory of a thrown juggling ball. The datasets we use are a simulation of a pong ball bouncing in a box, and a video of a person throwing a juggling ball. Unfortunately, we did not manage to achieve consistent good results on either of the tasks, though in both cases the GAN appeared to learn some relevant features of the movement of the object it was

[1]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[2]  Léon Bottou,et al.  Towards Principled Methods for Training Generative Adversarial Networks , 2017, ICLR.

[3]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[4]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[6]  Olof Mogren,et al.  C-RNN-GAN: Continuous recurrent neural networks with adversarial training , 2016, ArXiv.

[7]  Tom White,et al.  Sampling Generative Networks: Notes on a Few Effective Techniques , 2016, ArXiv.

[8]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[9]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[10]  Jost Tobias Springenberg,et al.  Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks , 2015, ICLR.

[11]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[12]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[13]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[14]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[15]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[16]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.