Notes

  • My notes (slides) during my PhD study in UMONS since Feb-2017

  • My style : step-by-step, crystal clear, from first principle

  • Mostly self-taught : email me if you catch a careless mistakes/typos

Tutorial/Lecture material

On Gradient Descent solving Quadratic problems On Coordinate Descent solving Quadratic problems (under construction)

Mathematical optimization

1st-order methods

Randomized 1st-order methods

2nd and higher order methods

Fixed point and splittings

Coordinate Descents

Non-convex Optimizations

Non-negative Matrix Factorizations : heuristics, algorithms and theory

Matrix Completion

Linear Algebra / Matrix Theory

Randomizations : Randomized Linear Algebra, Compressive Sensing, Random Matrix Theory

Multilinear/Tensor Algebra and Tensor methods

  • Fundamentals of Tensor

  • Tensor Multi-linear rank and rank

  • Why tensor : “easier” to get uniqueness of decomposition

  • Tensor method (3rd order method) for optimisation

  • Canonical Polyadic Decomposition

  • Non-negative Canonical Polyadic Decomposition

  • Tensor shortcut : \(kr(U,V)^\top kr(U,V) = U^\top U \odot V^\top V\)

  • The MTTKRP bottleneck

Projection operator

Paradigms in Machine Learning

Machine Learning applications

  • Hyper-spectral imaging

  • Audio source separation

  • Electricity load profiling

On software engineering in research

  • Version control

  • On efficient coding on experiments comparing multiple algorithms