Machine Learning Lab - projects:asgd https://wiki.eecs.yorku.ca/lab/MLL/ 2026-04-11T02:44:33+00:00 Machine Learning Lab https://wiki.eecs.yorku.ca/lab/MLL/ https://wiki.eecs.yorku.ca/lab/MLL/_media/wiki:dokuwiki-128.png text/html 2015-07-12T14:04:57+00:00 Anonymous (anonymous@undisclosed.example.com) Annealing SGD https://wiki.eecs.yorku.ca/lab/MLL/projects:asgd:start?rev=1436709897&do=diff Annealing SGD Annealed Stochastic Gradient Descent (AGD) Here, we propose a novel annealed gradient descent (AGD) method for deep learning. AGD optimizes a sequence of gradually improved smoother mosaic functions that approximate the original non-convex objective function according to an annealing schedule during optimization process. We present a theoretical analysis on its convergence properties and learning speed. The proposed AGD algorithm is applied to learning deep neural networks (…