Optimization

Convex and linear optimisation — standalone courses at UofT in the Operations Research stream.

Courses: Convex Optimisation, Linear Optimisation

Topics covered

• Linear programming: simplex method, interior-point methods, duality
• Convex functions, subgradients, epigraphs
• Lagrange multipliers and KKT conditions (constrained optimisation)
• Gradient descent variants (momentum, Adam) — direct connection to ML training
• Quadratic programs and second-order cone programming (introductory)

Why it matters

Optimisation is the engine of machine learning — every model trains by minimising a loss. Understanding the theory behind convergence, saddle points, and landscape geometry directly improves intuition for model design and hyperparameter tuning.

Related: Mathematics, ML, Numerical Methods