Andersen Ang | Papers | Talks | Notes | Misc.

...

Convex, strongly conve, Lipschitz, Lipschitz gradient, Lipschitz Hessian, lower semicontinuous function

1st-order Taylor approximation of convex function and Bregman divergence

Sandwich theorem for β-smooth function

Monotonictiy of gradient of strongly-convex smooth function

1st-order optimality condition for constrained optimization

Common glossaries in optimization

Coercive function

proj operator is firmly non-expensive & obtuse angle criterion

Augmented objective function & methods for constrained optimization

, and

Prox of norm, Moreau's decomposition, conjugate of norm = indicator function of dual norm

Characterizing subdifferential of norm

Projection onto nonnegative orthant, rectangular box and polyhedron | ball | ball | simplex | nonnegative unimodal vector, intersection of convex sets | -Lipschitz matrix | Spectraplex

Linear Optimization

Solving piece-wise linear equation

Motivation of mirror descent 1: Gradient vector lives in the dual space

Motivation of mirror descent 2: a geometry example

Mirror descent algorithm

Gradient flow gives gradient update, proximal point update, extra-gradient update and 4th order Runge-Kutta update

Convergence of gradient flow in continuous time

Convergence analysis of 1st-order method on a quadratic programming problem using dynamic system

Convergence of 4th-order Runge-Kutta update on least square problem

Deriving ODE model of Nesterov's accelerated gradient

Principle of least action & Euler-Lagrange equation of motion

Lagrangian Mutiplier

What is conjugate: ?

Properties of conjugate

Deriving the dual problem using conjugate

Duality | KKT | Slater’s constraint qualifications

Fast dual proximal gradient method

Fast primal-dual proximal gradient algorithm and preconditioning

How ADMM was developed

Convergence of ADMM

convergence rate of ADMM

ergodic convergence rate of ADMM

Where does operator splitting come from

Fixed point of average of non-expansive operator

Convergence of a proximal quasi-Newton's method

Cubic regularization

Sol to line search of Anderson Acceleration subproblem

Anderson Acceleration

Functions that has Lipschitz Hessian

Big finite sum

Confusing naming in the stochastic methods

Stochastic Gradient Method (SG)

MM: the technique of surrogate

Solving total variation norm regularised least square by MM

Convergence of MM

BSUM (only convergence, no rate)

TiTAN

Coordinate Descent - an introduction

Solving nonegative least sqaures with L1-regularization

Convergence of randomized block coordinate gradient descent on β-smooth convex function

Greedy coordinate descent

Accelerating coordinate descents

Polyak-Łojasiewicz inequality

Proximal Polyak-Łojasiewicz inequality

Kurdyka-Łojasiewicz property

Proximal Alternating Linearized Method (PALM)

Convergence of PALM on nonconvex problem, part 1 : generated sequence is non-increasing and the trajectory has finite length

Convergence of PALM on non-convex problem, part 2 : generated sequence converges to a critical point

Inertial Proximal Alternating Linearized Method (iPALM)

Xu-Yin's Block Coordinate Descent Method for Regularized Multiconvex Optimization, hand written

Error computation in matrix factorization

Hyperspectral unmixing & NMF

Short-time Fourier transform

ABC of tensors

Canonical Polyadic Decomposition: basic notations