Applied Linear Algebra and Least Squares
1 study materials · 9 sections
This course provides a practical introduction to linear algebra, focusing on the core concepts of vectors, matrices, and least squares methods. It emphasizes real-world applications such as machine learning, control systems, and data fitting while maintaining a streamlined mathematical framework. Students will explore how to represent data, solve systems of linear equations, and optimize models using QR factorization and iterative algorithms.
Course Sections
Vector Fundamentals and Inner Products
Key concepts: Scalar-vector multiplication · Unit vector · Sparse vector · Inner product · Complexity (flops)
Covers basic vector notation, scalar-vector multiplication, addition, and the definition of the inner product.
Linear Functions, Norms, and Distance
Key concepts: Superposition · Euclidean norm · Triangle inequality · Cauchy-Schwarz inequality · RMS value
Explores linear functions via superposition and measures of magnitude and distance using the Euclidean norm.
Clustering and Linear Independence
Key concepts: K-means algorithm · Linear independence · Basis · Orthonormal vectors · Gram-Schmidt algorithm
Introduces the K-means algorithm for data grouping and the concept of linear independence and bases.
Matrix Basics and Geometric Examples
Key concepts: Transpose · Matrix-vector multiplication · Incidence matrix · Convolution
Defines matrix dimensions, multiplication with vectors, and applications in geometry and graph theory.
Linear Equations and Dynamical Systems
Key concepts: Affine functions · Systems of linear equations · State-space model · Population dynamics
Covers the representation of systems of equations and the modeling of time-varying states.
Matrix Multiplication and Inverses
Key concepts: Matrix multiplication · QR Factorization · Left and Right Inverses · Pseudo-inverse
Explains matrix-matrix multiplication, QR factorization, and the conditions for matrix invertibility.
The Least Squares Problem and Data Fitting
Key concepts: Residual norm · Normal equations · Feature engineering · Overfitting · Least squares classifier
Focuses on minimizing the residual norm and applying least squares to regression and classification.
Regularization and Constrained Least Squares
Key concepts: Tikhonov regularization · Trade-off curves · KKT equations · Constrained least squares
Covers Tikhonov regularization for stability and solving least squares with equality constraints.
Nonlinear Least Squares
Key concepts: Gauss-Newton algorithm · Levenberg-Marquardt algorithm · Augmented Lagrangian · Penalty algorithm
Introduces iterative algorithms for solving least squares problems where the model is nonlinear.
Source Materials
Study Applied Linear Algebra and Least Squares on Lykke
This course page is free to open: signed out, you can read its first two sections, and signing in, on any plan, opens the rest. The free Lykke Chrome extension syncs your own Canvas courses, so ChatGPT, Claude and Muse can answer from your classes and deadlines. It only reads Canvas.
Add the free extensionSign in to open every section