My notes (slides) since my PhD study, email me if you catch a mistake/typo.
I update this page often, refresh your browser to avoid it showing the old version of this page.
Tutorial/Lecture material
On Gradient descent solving quadratic problems
[video]
Algebra
Continuous Optimization
General
Common glossaries in optimization
Coercive function
Projection operator is (firmly) nonexpansive and obtuse angle criterion (BourbakiCheneyGoldstein inequality)
Proximal operator is nonexpansive and firmly nonexpansive
Augmented Objective Function  an introduction
Penalty and Barrier Method, Interior point method
Fenchel conjugate of norm is indicator function on unit ball of dual norm
Subdifferential of norm \(\partial \ x \ = \left \{ v \in \mathbb{R}^n \Big  \langle v,x \rangle = \ x \, \ v \_* \leq 1 \right \}\)
Projection operator as proximal operator on indicator function
Project onto: nonnegative orthant, box and polyhedron, L2 ball, L1 ball, Lp ball, halfspace, simplex, monotonic vector, nonnegative unimodal vector, cone, intersection of convex sets, \(L\)Lipschitz matrix, Spectraplex
Nonnegative Matrix Factorizations : heuristics, algorithms, theory
Matrix Completion
Theory
Algorithm
MC by MajorizationMinimization, by Proximal Point Algorithm / primaldual method, by Augmented Lagrangian Method, by DouglasRachford spitting algorithm, by ADMM, by Iterative Reweighted Least Squares
Linear Algebra / Matrix Theory
Randomized Linear Algebra, Compressive Sensing, Random Matrix
Multilinear/Tensor Algebra and Tensor methods
Machine Learning
Signal Processing
Miscellaneous items
On software engineering
