5. Eigenvalues
Table of contents
This guide is incomplete and is under active development.
Eigenvalues and eigenvectors
Eigenvalues and eigenvectors appear often in data science. We’ll start by establishing their fundamentals; the applications will become clear as the semester progresses.
\(\lambda_i\) is an eigenvalue of square matrix \(A\), corresponding to the eigenvector \(\vec{v}_i\), if:
\[A \vec{v}_i = \lambda_i \vec{v}_i\]In other words, \(\vec{v}_i\) is an eigenvalue of \(A\) if, when left-multiplied by \(A\), its direction doesn’t change, only its length. The amount \(\vec{v}_i\)’s length is scaled by is \(\lambda_i\). We also say that \(\lambda_i\) and \(\vec{v}_i\) form an eigenvalue-eigenvector pair of \(A\). (Note that \(\vec{0}\) is never considered to be an eigenvector, since any matrix times \(\vec{0}\) is always just \(\vec{0}\).)
For example, if \(A = \begin{bmatrix} -5 & 2 \\ -7 & 4 \end{bmatrix}\), then \(\vec{v}_1 = \begin{bmatrix} 2 \\ 7 \end{bmatrix}\) is an eigenvector of \(A\) corresponding to the eigenvalue \(\lambda_1 = 2\), because:
\[A\vec{v}_1 = \begin{bmatrix} -5 & 2 \\ -7 & 4 \end{bmatrix} \begin{bmatrix} 2 \\ 7 \end{bmatrix} = \begin{bmatrix} -5(2) + 2(7) \\ -7(2) + 4(7) \end{bmatrix} = \begin{bmatrix} 4 \\ 14 \end{bmatrix} = 2 \begin{bmatrix} 2 \\ 7 \end{bmatrix} = 2 \vec{v}_1\]So, when \(\vec{v}_1\) is multiplied by \(A\), it still points in the same direction, it’s just doubled in length.
Verify yourself that \(\lambda_2 = -3\) is also an eigenvalue of \(A\), corresponding to the eigenvector \(\vec{v}_2 = \begin{bmatrix} 1 \\ 1\end{bmatrix}\).
Invertibility and rank
Fact: An \(n \times n\) matrix can have at most \(n\) non-zero eigenvalues. The rank of a square matrix is equal to the number of non-zero eigenvalues it has.
Okay, so what did we need all of that for? It’s to use this fact:
More coming soon!
Read more about eigenvalues and eigenvectors here, or watch this Khan Academy video.