James Martens

发表

On the importance of initialization and momentum in deep learning

Geoffrey E. Hinton, Ilya Sutskever, George E. Dahl, 2013, ICML.

Adding Gradient Noise Improves Learning for Very Deep Networks pdf

Quoc V. Le, Lukasz Kaiser, Ilya Sutskever, 2015, ArXiv.

Training Deep and Recurrent Networks with Hessian-Free Optimization

Ilya Sutskever, James Martens, Ilya Sutskever, 2012, Neural Networks: Tricks of the Trade.

On the Representational Efficiency of Restricted Boltzmann Machines

R. Zemel, James Martens, Arkadev Chattopadhya, 2013, NIPS 2013.

Estimating the Hessian by Back-propagating Curvature

Ilya Sutskever, Kevin Swersky, James Martens, 2012, ICML.

Deep learning via Hessian-free optimization

James Martens, James Martens, 2010, ICML.

Learning Recurrent Neural Networks with Hessian-Free Optimization

Ilya Sutskever, James Martens, Ilya Sutskever, 2011, ICML.

Generating Text with Recurrent Neural Networks

Geoffrey E. Hinton, Ilya Sutskever, James Martens, 2011, ICML.

Optimizing Neural Networks with Kronecker-factored Approximate Curvature

Roger B. Grosse, James Martens, James Martens, 2015, ICML.

New Insights and Perspectives on the Natural Gradient Method

James Martens, James Martens, 2014, J. Mach. Learn. Res..

On the Expressive Efficiency of Sum Product Networks pdf

James Martens, Venkatesh Medabalimi, James Martens, 2014, ArXiv.

Stochastic Gradient Langevin dynamics that Exploit Neural Network Structure

Jasper Snoek, Roger B. Grosse, David Duvenaud, 2018, ICLR.

Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

Guodong Zhang, Roger B. Grosse, Sushant Sachdeva, 2019, NeurIPS.

Differentiable Game Mechanics

Thore Graepel, Karl Tuyls, Jakob N. Foerster, 2019, J. Mach. Learn. Res..

On the Expressive Power of Restricted Boltzmann Machines

Toniann Pitassi, Richard S. Zemel, Arkadev Chattopadhyay, 2013, NIPS.

Learning the Linear Dynamical System with ASOS

James Martens, James Martens, 2010, ICML.

The Mechanics of n-Player Differentiable Games pdf

Thore Graepel, Karl Tuyls, Jakob N. Foerster, 2018, ICML.

Normalization for probabilistic inference with neurons

Chris Eliasmith, James Martens, James Martens, 2011, Biological Cybernetics.

Distributed Second-Order Optimization using Kronecker-Factored Approximations

Roger B. Grosse, Jimmy Ba, James Martens, 2016, ICLR.

Pre-training via Denoising for Molecular Property Prediction

James Martens, Y. Teh, P. Battaglia, 2022, ArXiv.

Second-order Optimization for Neural Networks

James Martens, 2016 .

Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks

James Martens, Guodong Zhang, R. Grosse, 2019, NeurIPS.

Pre-training via Denoising for Molecular Property Prediction

James Martens, Y. Teh, P. Battaglia, 2022, ICLR.

Kronecker-factored Curvature Approximations for Recurrent Neural Networks

Jimmy Ba, James Martens, Matt Johnson, 2018, ICLR.

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation

James Martens, Y. Teh, Samuel L. Smith, 2023, ICLR.

Adversarial Robustness through Local Linearization

Pushmeet Kohli, Sven Gowal, Krishnamurthy Dvijotham, 2019, NeurIPS.

Generating Text with Recurrent Neural Networks

Geoffrey E. Hinton, Ilya Sutskever, James Martens, 2011, ICML.

A Kronecker-factored approximate Fisher matrix for convolution layers

Roger B. Grosse, James Martens, James Martens, 2016, ICML.

Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers

James Martens, Aleksandar Botev, Guodong Zhang, 2022, ICLR.

Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping pdf

Samuel S. Schoenholz, Valentin Dalibard, Guillaume Desjardins, 2021, ArXiv.

P RE - TRAINING VIA D ENOISING FOR M OLECULAR P ROPERTY P REDICTION

James Martens, Y. Teh, P. Battaglia, 2023 .

New perspectives on the natural gradient method pdf

James Martens, James Martens, 2014, ArXiv.

On the validity of kernel approximations for orthogonally-initialized neural networks pdf

James Martens, James Martens, 2021, ArXiv.

On the Variance of Unbiased Online Recurrent Optimization pdf

James Martens, Tim Cooijmans, James Martens, 2019, ArXiv.

Adding Gradient Noise Improves Learning for Very Deep Networks

Quoc V. Le, Lukasz Kaiser, James Martens, 2015, ArXiv.