一阶机器学习优化算法Paper汇总

对一阶机器学习优化算法ADMM、Coordinate Descent和Gradient Descent的paper加以简单整理。

ADMM

• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

• Alternating Direction Method of Multipliers (PPT)

• LASSO using ADMM

• Alternating direction method of multipliers

• L1 logistic regression ADMM demo

• 用ADMM实现统计学习问题的分布式计算

• 分布式计算、统计学习与ADMM算法

Coordinate Descent

• Coordinate Descent method(PPT)

• Coordinate descent(PPT)

• An Improved GLMNET for L1-regularized Logistic Regression

• Coordinate Descent Algorithms

• A Comparison of Optimization Methods and Software for Large-scale L1-regularized Linear Classification

• COORDINATE DESCENT ALGORITHMS FOR LASSO PENALIZED REGRESSION

• SVM Coordinate demo

parallel

• Parallel Coordinate Descent for L1-Regularized Loss Minimization
• Parallel Dual Coordinate Descent Method for Large-scale Linear Classification in Multi-core Environments(SVM parallel)

Gradient Descent

• Logistic Regression and Gradient Descent(PPT)

• Stochastic Gradient Descent Tricks

variants

• Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
• ADADELTA: AN ADAPTIVE LEARNING RATE METHOD
• ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION
• RMSprop(ppt)
• Unit Tests for Stochastic Optimization
  parallel
• Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Delay-Tolerant Algorithms for Asynchronous Distributed Online Learning
• Large Scale Distributed Deep Networks(Downpour SGD)

• Deep learning with Elastic Averaging SGD

• An overview of gradient descent optimization algorithms

Summary

• Optimization (and Learning)