Applied Linear Algebra and Least Squares

View original course

1 study materials · 9 sections

This course provides a practical introduction to linear algebra, focusing on the core concepts of vectors, matrices, and least squares methods. It emphasizes real-world applications such as machine learning, control systems, and data fitting while maintaining a streamlined mathematical framework. Students will explore how to represent data, solve systems of linear equations, and optimize models using QR factorization and iterative algorithms.

Course Sections

Vector Fundamentals and Inner Products

Key concepts: Scalar-vector multiplication · Unit vector · Sparse vector · Inner product · Complexity (flops)

Covers basic vector notation, scalar-vector multiplication, addition, and the definition of the inner product.

Linear Functions, Norms, and Distance

Key concepts: Superposition · Euclidean norm · Triangle inequality · Cauchy-Schwarz inequality · RMS value

Explores linear functions via superposition and measures of magnitude and distance using the Euclidean norm.

Clustering and Linear Independence

Key concepts: K-means algorithm · Linear independence · Basis · Orthonormal vectors · Gram-Schmidt algorithm

Introduces the K-means algorithm for data grouping and the concept of linear independence and bases.

Matrix Basics and Geometric Examples

Key concepts: Transpose · Matrix-vector multiplication · Incidence matrix · Convolution

Defines matrix dimensions, multiplication with vectors, and applications in geometry and graph theory.

Linear Equations and Dynamical Systems

Key concepts: Affine functions · Systems of linear equations · State-space model · Population dynamics

Covers the representation of systems of equations and the modeling of time-varying states.

Matrix Multiplication and Inverses

Key concepts: Matrix multiplication · QR Factorization · Left and Right Inverses · Pseudo-inverse

Explains matrix-matrix multiplication, QR factorization, and the conditions for matrix invertibility.

The Least Squares Problem and Data Fitting

Key concepts: Residual norm · Normal equations · Feature engineering · Overfitting · Least squares classifier

Focuses on minimizing the residual norm and applying least squares to regression and classification.

Regularization and Constrained Least Squares

Key concepts: Tikhonov regularization · Trade-off curves · KKT equations · Constrained least squares

Covers Tikhonov regularization for stability and solving least squares with equality constraints.

Nonlinear Least Squares

Key concepts: Gauss-Newton algorithm · Levenberg-Marquardt algorithm · Augmented Lagrangian · Penalty algorithm

Introduces iterative algorithms for solving least squares problems where the model is nonlinear.

Source Materials

Study Applied Linear Algebra and Least Squares on Lykke

This course page is free to open: signed out, you can read its first two sections, and signing in, on any plan, opens the rest. The free Lykke Chrome extension syncs your own Canvas courses, so ChatGPT, Claude and Muse can answer from your classes and deadlines. It only reads Canvas.

Add the free extensionSign in to open every section

Browse more course pages

Applied Linear Algebra and Least Squares | Lykke