$$ \newcommand{\mybold}[1]{\boldsymbol{#1}} \newcommand{\trans}{\intercal} \newcommand{\norm}[1]{\left\Vert#1\right\Vert} \newcommand{\abs}[1]{\left|#1\right|} \newcommand{\bbr}{\mathbb{R}} \newcommand{\bbz}{\mathbb{Z}} \newcommand{\bbc}{\mathbb{C}} \newcommand{\gauss}[1]{\mathcal{N}\left(#1\right)} \newcommand{\chisq}[1]{\mathcal{\chi}^2_{#1}} \newcommand{\studentt}[1]{\mathrm{StudentT}_{#1}} \newcommand{\fdist}[2]{\mathrm{FDist}_{#1,#2}} \newcommand{\argmin}[1]{\underset{#1}{\mathrm{argmin}}\,} \newcommand{\projop}[1]{\underset{#1}{\mathrm{Proj}}\,} \newcommand{\proj}[1]{\underset{#1}{\mybold{P}}} \newcommand{\expect}[1]{\mathbb{E}\left[#1\right]} \newcommand{\prob}[1]{\mathbb{P}\left(#1\right)} \newcommand{\dens}[1]{\mathit{p}\left(#1\right)} \newcommand{\var}[1]{\mathrm{Var}\left(#1\right)} \newcommand{\cov}[1]{\mathrm{Cov}\left(#1\right)} \newcommand{\sumn}{\sum_{n=1}^N} \newcommand{\meann}{\frac{1}{N} \sumn} \newcommand{\cltn}{\frac{1}{\sqrt{N}} \sumn} \newcommand{\trace}[1]{\mathrm{trace}\left(#1\right)} \newcommand{\diag}[1]{\mathrm{Diag}\left(#1\right)} \newcommand{\grad}[2]{\nabla_{#1} \left. #2 \right.} \newcommand{\gradat}[3]{\nabla_{#1} \left. #2 \right|_{#3}} \newcommand{\fracat}[3]{\left. \frac{#1}{#2} \right|_{#3}} \newcommand{\W}{\mybold{W}} \newcommand{\w}{w} \newcommand{\wbar}{\bar{w}} \newcommand{\wv}{\mybold{w}} \newcommand{\X}{\mybold{X}} \newcommand{\x}{x} \newcommand{\xbar}{\bar{x}} \newcommand{\xv}{\mybold{x}} \newcommand{\Xcov}{\Sigmam_{\X}} \newcommand{\Xcovhat}{\hat{\Sigmam}_{\X}} \newcommand{\Covsand}{\Sigmam_{\mathrm{sand}}} \newcommand{\Covsandhat}{\hat{\Sigmam}_{\mathrm{sand}}} \newcommand{\Z}{\mybold{Z}} \newcommand{\z}{z} \newcommand{\zv}{\mybold{z}} \newcommand{\zbar}{\bar{z}} \newcommand{\Y}{\mybold{Y}} \newcommand{\Yhat}{\hat{\Y}} \newcommand{\y}{y} \newcommand{\yv}{\mybold{y}} \newcommand{\yhat}{\hat{\y}} \newcommand{\ybar}{\bar{y}} \newcommand{\res}{\varepsilon} \newcommand{\resv}{\mybold{\res}} \newcommand{\resvhat}{\hat{\mybold{\res}}} \newcommand{\reshat}{\hat{\res}} \newcommand{\betav}{\mybold{\beta}} \newcommand{\betavhat}{\hat{\betav}} \newcommand{\betahat}{\hat{\beta}} \newcommand{\betastar}{{\beta^{*}}} \newcommand{\bv}{\mybold{\b}} \newcommand{\bvhat}{\hat{\bv}} \newcommand{\alphav}{\mybold{\alpha}} \newcommand{\alphavhat}{\hat{\av}} \newcommand{\alphahat}{\hat{\alpha}} \newcommand{\omegav}{\mybold{\omega}} \newcommand{\gv}{\mybold{\gamma}} \newcommand{\gvhat}{\hat{\gv}} \newcommand{\ghat}{\hat{\gamma}} \newcommand{\hv}{\mybold{\h}} \newcommand{\hvhat}{\hat{\hv}} \newcommand{\hhat}{\hat{\h}} \newcommand{\gammav}{\mybold{\gamma}} \newcommand{\gammavhat}{\hat{\gammav}} \newcommand{\gammahat}{\hat{\gamma}} \newcommand{\new}{\mathrm{new}} \newcommand{\zerov}{\mybold{0}} \newcommand{\onev}{\mybold{1}} \newcommand{\id}{\mybold{I}} \newcommand{\sigmahat}{\hat{\sigma}} \newcommand{\etav}{\mybold{\eta}} \newcommand{\muv}{\mybold{\mu}} \newcommand{\Sigmam}{\mybold{\Sigma}} \newcommand{\rdom}[1]{\mathbb{R}^{#1}} \newcommand{\RV}[1]{\tilde{#1}} \def\A{\mybold{A}} \def\A{\mybold{A}} \def\av{\mybold{a}} \def\a{a} \def\B{\mybold{B}} \def\S{\mybold{S}} \def\sv{\mybold{s}} \def\s{s} \def\R{\mybold{R}} \def\rv{\mybold{r}} \def\r{r} \def\V{\mybold{V}} \def\vv{\mybold{v}} \def\v{v} \def\U{\mybold{U}} \def\uv{\mybold{u}} \def\u{u} \def\W{\mybold{W}} \def\wv{\mybold{w}} \def\w{w} \def\tv{\mybold{t}} \def\t{t} \def\Sc{\mathcal{S}} \def\ev{\mybold{e}} \def\Lammat{\mybold{\Lambda}} $$

Linear Algebra topics

\(\LaTeX\)

Summary of prerequisite requirements

For 151A, we require familiarity with the following basic concepts from linear algebra. In rough order of importance,

• Essential: Basic vector and matrix operations
  • Addition and subtraction
  • Multiplication by scalars
  • Matrix-matrix and matrix-vectror multiplication
  • Vector dot products, norms, and orthogonality
  • Matrix transposes and inverses, symmetric matrices
• Essential: Linear systems
  • Representing a linear system as a matrix equation and vice-versa
  • Solving exactly specified linear systems using matrix inverses
• Very important: Vector spaces and bases
  • Rank and null spaces of matrices
  • Solution space of linear systems that are not exactly identified
  • Orthogonal vectors and orthonormal bases
• Helpful: Eigendecomposition of symmetric, square matrices
  • Eigenvalues and eigenvectors
  • The relationship between an inverse and the eigendecomposition
  • Eigenvectors as an orthonormal basis
  • The trace and determinant as a function of eigenvalues
  • Positive definiteness

There are some (wonderful, useful, fascinating) linear algebra topics that we won’t require in 151A, but which are commonly included in an introductory linear algebra course:

• Not very important for 151A (but great to know anyway):
  • Manually solving systems of equations (e.g., row-echelon form, permutations)
  • Advanced properties of determinants (e.g. Cramer’s rule)
  • Singular value decompositions
  • Linear operators on abstract spaces (e.g. spaces of polynomials)
  • Eigendecompositions of non-symmetric matrices
  • Factorizations (e.g. LU, QR)
  • Gram-Schmidt orthogonalization (other than knowing that it can be done)
  • Jordan canonical forms
  • Matrix exponentiation and power series

Introduction to Applied Linear Algebra by Boyd and Vandenberghe

A good book which is freely available online is Introduction to Applied Linear Algebra by Boyd and Vandenberghe (B&V). For 151A, the following sections and their exercises are useful:

• Vector operations
  • B&V 1.1-1.4 (addition, scalar multiplication, dot product)
  • B&V 3.1, 3.2 (norms — we will mostly use the Euclidian (L2) norm)
• Matrix operations
  • B&V 6.1-6.4 (addition, zero and identity, trace, transpose)
  • B&V 10.1 (multiplication)
  • B&V 11.1-11.2 (inverses)
• Linear systems
  • B&V 8.1, 8.2.2, 8.3 (definition and examples)
  • B&V 11.3 (solution via matrix inversions)
• Vector spaces and bases
  • B&V 5.1-5.3 (linear depenAbsolutely necessarydence, bases, orthonormality)

Some more advanced topics not treaded in B&V are eigenvalues and nullspaces.

MIT Open Courseware with Gilbert Strang

There are good recorded lectures in the MIT open courseware linear algebra series. The corresponding textbook, Strang’s Introduction to Linear Algebra, is also good, but is unfortunately not available online through the library website, and there are only a few hard copies available.

Assuming you can get your hands on a copy of the book, here is what I recommend reviewing. The sections refer to readings. The sections in the videos do not appear to line up perfectly with the readings, so where they’ve deviated, I’ve made my best guess as to the corresponding video.

• Essential: Sections 3, 6, 10, 14
• Very important: Sections 7, 15, 16, 10 (video 9), 11 (video 9)
• Very helpful: Sections 17, 21, 27 (video 25), 28 (video 25), 33 (video 30)

Yu Tsumura’s website

The website by Yu Tsumura has a wealth of interesting problems with solutions. Based on the titles, the following sections could be useful for 151A:

• Introduction to Matrices
• Elementary Row Operations
• Solutions of Systems of Linear Equations
• Linear Combination and Linear Independence
• Nonsingular Matrices
• Inverse Matrices
• Subspaces in Rn
• Bases and Dimension of Subspaces in Rn
• Bases and Coordinate Vectors
• Linear Transformation from Rn to Rm
• Orthogonal Bases
• Introduction to Eigenvalues and Eigenvectors
• Eigenvectors and Eigenspaces
• Dot Products and Length of Vectors