Math 512 Notes
Winter 2001
Numerical Linear Algebra: Trefethen and Bau
Lecture 1 – Matrix Vector Multiplication
Concepts:
- Matrix as a linear operator
- Polynomial as a matrix vector product: (1 by n) matrix is
- Matrix-matrix product B=AC in terms of
- Inner product
- Matrix vector product
- Outer product
(rank one matrix)
- Range(A)= column space of A
- Rank of matrix = dimension of Range(A) = # independent columns of A = # independent rows of A
- An m-by-n matrix A with m>=n has full rank if and only if it maps no two distinct vectors to the same vector
- Equivalent properties of a square matrix on invertibility
- Think of
as the vector s.t. Ax=b: so x is the vector of coefficients of the expansion of b in the basis columns of A è premultiply by inverse is just changing basis !!
Lecture 2 : Orthogonal vectors and Matrices
Concepts:
- Hermitian (conjugate) of A: A* = conjugate transpose of A = Adjoint of A
- Real matrix: Hermitian= transpose = adjoint
- Symmetric matrix: real matrix that is same as its transpose (adjoint )
- Inner product and Euclidean length of complex vectors
- Angle between vectors: geometric view
- Inner product is bilinear
- Properties of transpose and inverse: (AB)T=BTAT, etc
- Orthogonal vectors
- Elements of an orthogonal set are piecewise orthogonal
- Theorem: Vectors in an orthogonal set are linearly independent
- Orthogonal decomposition of vector
- Unitary matrix (orthogonal in real case): square matrix whose columns form an orthonormal basis
- Angle and length invariance of inner product after unitary transform
Lecture 3 – Norms
Concepts
- Abstract definition of a vector norm
- P-norms and related circles
- Weighted p-norms
- Induced matrix norms
- P-norm of diagonal matrices
- 1-norm and infinite norm
- Holder inequality and Cauchy-Schwarz inequality (statement only)
- Consequence of induced norm: ||AB||<=||A|| ||B||
- Trace of matrix = sum of diagonal entries = sum of eigenvalues
- Frobenius norm as square root of trace of A*A
- Frobenius norm also has matrix product bounding properties
- 2-Norm and Frobenius norm invariance under unitary transform
Lecture 4 – SVD
Concept:
- Geometric view: Matrix maps unit sphere to a hyperellipse
- Singular values and (left and right) singular vectors
- Reduced SVD – general form
- Full SVD – general form
- Existence and Uniqueness theorem of SVD