What is orthogonality?
Two vectors v, w in an inner product space V are orthogonal if their inner product is zero. <v, w>=0. An orthonormal basis, therefore, is an orthogonal basis where each vector is unit norm. So, going back to our inner product notes, we can use the Gram-Schmidt process to convert a linearly independent set into an orthonormal basis.
What is vector projection?
A projection of a vector v onto a unit vector e is given by: .
What is an orthogonal matrix?
An orthogonal matrix is a square matrix whose rows & columns are orthogonal unit vectors, so the column space of an orthogonal matrix is an orthonormal basis. The inverse of an orthogonal matrix, therefore, is its transpose. Furthermore, an orthogonal matrix preserves the Euclidian length of vectors, as it does no stretch or scaling of the input vector. In other words: . Furthermore, it will always have a determinant of +/- 1. Finally, orthogonal matrices, as linear transformations, represent transformations which either rotate or reflect in space. Note: The product of any two orthogonal matrices is orthogonal as well.
What is QR factorization?
Any non-singular matrix A can be factorized as: . Where Q is an orthogonal matrix and R is an upper triangular matrix. Every collection of non-zero orthogonal vectors form a basis for its span. This is done as so: