16 - Projections and Least square fitting of straight line

• if b in column space of A, then Pb = b

• if b ⊥ column space, then Pb = 0, since b in null space

• Show example of fitting straight line

• If A has independent columns, then A^TA is invertible
  • To prove above statement, prove the null space of A^TA contains only zero vector
  • A^TAx = 0
  • x^TA^TAx = 0
  • (Ax)^TAx = 0
  • Ax = 0
  • x = 0
• Columns are definitely independent if they are perpendicular vectors

17 - Orthogonal basis, Orthogonal square matrix (Q), Gram-Schmidt (A –> Q)

• Orthonormal vector -
  • q_i^Tq_j = 0, if i is not equal to j
  • q_i^Tq_j = 1, if i is equal to j
• Q^TQ = I
• if Q is square - Q^T = Q^-1
• What is made easier?
  • Projection matrix calculation
  • P = Q(Q^TQ)^-1Q^T = QQ^T, I if Q is square
• Gram-Schmidt (make matrix A to Q)
  • [Diagram]
  • take independent vectors of A - a, b
  • Make them orthogonal by projection
  • Normalize for orthonormal
  • For multiple vectors - one by one take vector and remove components of previous vectors from it
  • [Diagram]

18 - Determinants, Properties

1. det(I) = 1
2. row exchange reverses the sign of determinant
3. [Diagram]
   1. Multiplier from row can be taken common and determinant will be multiplied by the same
   2. Determinant can be split with respect to one row
4. 2 equal or dependent rows –> det = 0
5. Subtract l x row_i from row_k –> determinant doesn’t change
6. row of 0’s –> det = 0
7. upper triangular –> det = product of diagonal entries
8. det(A) = 0 ↔ A is singular OR det(A) ≠ 0 ↔ A is invertible
9. det(AB) = det(A)*det(B)
10. det(A^T) = det(A)

**All above properties also holds for column

• determinant of skew-symmetric matrix = 0 (use rule 10)

19 - Determinant Formula, Cofactors formula, Tridiagonal matrices

• Use rule 3.2 of determinant properties

• n! terms

• Cofactors - connect nxn determinant to (n-1)x(n-1)

• det(A) = a₁₁C₁₁ + a₁₂C₁₂ + ….. + a_1nC_1n

• Tridiagonal matrices [Diagram]
• det(Aⁿ) = det(A^n-1) - det(A^n-2)

20 - Formula for A^-1, Cramer’s rule, det(A) = Volume of box

• AC^T = det(A) I

• A-1 = (1 /

  A

  ) CT….C = Cofactor matrix

• row_i * Cofactors_j = 0 …..if i != j
  • Hint: Identical rows
• Cramer’s Rule:
  • Used for solving Ax = b
  • x = A^-1b

  • x = (1 /

    A

    ) CTb

  • x₁ = det(B₁) / det(A), x₂ = det(B₂) / det(A)
    • B_j = Matrix A with j^th column substituted by b
• Determinant of 3x3 matrix = Volume of space enclosed by the column vectors (Refer 3b1b video for visualization)