Eigenvalue

Let $A$ be a linear transformation represented by a Matrix A. If there is a Vector ${\bf X}\in\Bbb{R}^n \not= {\bf0}$ such that

$${\hbox{\sf A}}{\bf X} = \lambda {\bf X}$$ (1)

for some Scalar $\lambda$, then $\lambda$ is the eigenvalue of A with corresponding (right) Eigenvector ${\bf X}$. Letting A be a $k\times k$ Matrix,

$$\left[{\matrix{a_{11} & a_{12} & \cdots & a_{1k}\cr a_{21} ... ...ots & \vdots\cr a_{k1} & a_{k2} & \cdots & a_{kk}\cr}}\right]$$ (2)

with eigenvalue $\lambda$, then the corresponding Eigenvectors satisfy

$$\left[{\matrix{a_{11} & a_{12} & \cdots & a_{1k}\cr a_{21} ... ...lambda \left[{\matrix{x_1\cr x_2\cr \vdots\cr x_k\cr}}\right],$$ (3)

which is equivalent to the homogeneous system

$$\left[{\matrix{a_{11}-\lambda & a_{12} & \cdots & a_{1k}\cr ... ...}}\right] = \left[{\matrix{0\cr 0\cr \vdots\cr 0\cr}}\right].$$ (4)

Equation (4) can be written compactly as

$$({\hbox{\sf A}}-\lambda{\hbox{\sf I}}){\bf X}={\bf0},$$ (5)

where I is the Identity Matrix.

As shown in Cramer's Rule, a system of linear equations has nontrivial solutions only if the Determinant vanishes, so we obtain the Characteristic Equation

$$\vert{\hbox{\sf A}}-\lambda{\hbox{\sf I}}\vert=0.$$ (6)

If all $k$ $\lambda$s are different, then plugging these back in gives $k-1$ independent equations for the $k$ components of each corresponding Eigenvector. The Eigenvectors will then be orthogonal and the system is said to be nondegenerate. If the eigenvalues are $n$-fold Degenerate, then the system is said to be degenerate and the Eigenvectors are not linearly independent. In such cases, the additional constraint that the Eigenvectors be orthogonal,

$${\bf X}_i\cdot {\bf X}_j = X_iX_j \delta_{ij},$$ (7)

where $\delta_{ij}$ is the Kronecker Delta, can be applied to yield $n$ additional constraints, thus allowing solution for the Eigenvectors.

Assume A has nondegenerate eigenvalues $\lambda_1, \lambda_2, \ldots, \lambda_n$ and corresponding linearly independent Eigenvectors ${\bf X}_1, {\bf X}_2, \ldots, {\bf X}_k$ which can be denoted

$$\left[{\matrix{x_{11}\cr x_{12}\cr \vdots\cr x_{1k}\cr}}\rig... ...eft[{\matrix{x_{k1}\cr x_{k2}\cr \vdots\cr x_{kk}\cr}}\right].$$ (8)

Define the matrices composed of eigenvectors

$${\hbox{\sf P}} \equiv \left[{\matrix{{\bf X}_1 & {\bf X}_2 &... ...ots & \vdots\cr x_{1k} & x_{2k} & \cdots & x_{kk}\cr}}\right]$$ (9)

and eigenvalues

$${\hbox{\sf D}} \equiv \left[{\matrix{\lambda_1 & 0 & \cdots ... ...dots & \ddots & \vdots\cr 0 & 0 & \cdots & \lambda_k}}\right],$$ (10)

where ${\hbox{\sf D}}$ is a Diagonal Matrix. Then

$${\hbox{\sf A}}{\hbox{\sf P}} = {\hbox{\sf A}}\left[\begin{array}{cccc}{\bf X}_1 & {\bf X}_2 & \cdots & {\bf X}_k\end{array}\right]$$

$$= \left[\begin{array}{cccc}{\hbox{\sf A}}{\bf X}_1 & {\hbox{\sf A}}{\bf X}_2 & \cdots & {\hbox{\sf A}}{\bf X}_k\end{array}\right]$$

$$= \left[\begin{array}{cccc}\lambda_1{\bf X}_1 & \lambda_2{\bf X}_2 & \cdots & \lambda_k{\bf X}_k\end{array}\right]$$

$$= \left[\begin{array}{cccc}\lambda_1x_{11} & \lambda_2x_{21} & \cdo... ... \lambda_1x_{1k} & \lambda_2x_{2k} & \cdots & \lambda_kx_{kk}\end{array}\right]$$

$$= \left[\begin{array}{cccc}x_{11} & x_{21} & \cdots & x_{k1}\\ x_{... ...dots & \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & \lambda_k\end{array}\right]$$

$$= {\hbox{\sf P}}{\hbox{\sf D}},$$ (11)

so

$${\hbox{\sf A}} = {\hbox{\sf P}}{\hbox{\sf D}}{\hbox{\sf P}}^{-1}.$$ (12)

Furthermore,

$${\hbox{\sf A}}^2 = ({\hbox{\sf P}}{\hbox{\sf D}}{\hbox{\sf P}}^{-1})({\hbox{\sf P}}{... ...box{\sf D}}({\hbox{\sf P}}^{-1}{\hbox{\sf P}}){\hbox{\sf D}}{\hbox{\sf P}}^{-1}$$

$$= {\hbox{\sf P}}{\hbox{\sf D}}^2{\hbox{\sf P}}^{-1}.$$ (13)

By induction, it follows that for $n>0$,

$${\hbox{\sf A}}^n={\hbox{\sf P}}{\hbox{\sf D}}^n{\hbox{\sf P}}^{-1}.$$ (14)

The inverse of A is

$${\hbox{\sf A}}^{-1} = ({\hbox{\sf P}}{\hbox{\sf D}}{\hbox{\s... ...)^{-1} = {\hbox{\sf P}}{\hbox{\sf D}}^{-1}{\hbox{\sf P}}^{-1},$$ (15)

where the inverse of the Diagonal Matrix D is trivially given by

$${\hbox{\sf D}}^{-1}= {1\over k} \left[{\matrix{{\lambda_1}^... ...\ddots & \vdots\cr 0 & 0 & \cdots & {\lambda_k}^{-1}}}\right].$$ (16)

Equation (14) therefore holds for both Positive and Negative $n$.

A further remarkable result involving the matrices ${\hbox{\sf P}}$ and ${\hbox{\sf D}}$ follows from the definition

$$e^{\hbox{{\sf A}}} \equiv \sum_{n=0}^\infty {{\hbox{\sf A}}^n\over n!} = \sum_{n=0}^\infty {{\hbox{\sf P}}{\hbox{\sf D}}^n{\hbox{\sf P}}^{-1}\over n!}$$

$$= {\hbox{\sf P}}\left({\sum_{n=0}^\infty {\hbox{\sf D}}^n\over n!}\right){\hbox{\sf P}}^{-1} = {\hbox{\sf P}}e^{\hbox{{\sf D}}}{\hbox{\sf P}}^{-1}.$$ (17)

Since D is a Diagonal Matrix,

$$e^{\hbox{{\sf D}}} = \sum_{n=0}^\infty {{\hbox{\sf D}}^n\over n!} = \sum_{n=0}^\infty ... ... & \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & {\lambda_k}^n\end{array}\right]$$

$$= \left[\begin{array}{cccc}\sum_{n=0}^\infty {{\lambda_1}^n\over n!... ...\ 0 & 0 & \cdots & \sum_{n=0}^\infty {{\lambda_k}^n\over n!}\end{array}\right]$$

$$= \left[\begin{array}{cccc}e^{\lambda_1} & 0 & \cdots & 0\\ 0 & e^... ...& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & e^{\lambda_k}\end{array}\right],$$ (18)

$e^{\hbox{{\sf D}}}$ can be found using

$${\hbox{\sf D}}^n = \left[{\matrix{{\lambda_1}^n & 0 & \cdots... ...\ddots & \vdots\cr 0 & 0 & \cdots & {\lambda_k}^n\cr}}\right].$$ (19)

Assume we know the eigenvalue for

$${\hbox{\sf A}}\bf {X}=\lambda\bf {X}.$$ (20)

Adding a constant times the Identity Matrix to A,

$$({\hbox{\sf A}}+c\,{\hbox{\sf I}}){\bf X} = (\lambda+c){\bf X} \equiv \lambda'\bf {X},$$ (21)

so the new eigenvalues equal the old plus $c$. Multiplying A by a constant $c$

$$(c\,{\hbox{\sf A}}){\bf X} = c\,(\lambda {\bf X}) \equiv \lambda'\bf {X},$$ (22)

so the new eigenvalues are the old multiplied by $c$.

Now consider a Similarity Transformation of A. Let $\vert{\hbox{\sf A}}\vert$ be the Determinant of A, then

$$\vert{\hbox{\sf Z}}^{-1}{\hbox{\sf A}}{\hbox{\sf Z}}-\lambda {\hbox{\sf I}}\vert = \vert{\hbox{\sf Z}}^{-1}({\hbox{\sf A}}-\lambda {\hbox{\sf I}}){\hbox{\sf Z}}\vert$$

$$= \vert{\hbox{\sf Z}}\vert \,\vert{\hbox{\sf A}}-\lambda {\hbox{\sf... ...vert{\hbox{\sf Z}}^{-1}\vert = \vert{\hbox{\sf A}}-\lambda {\hbox{\sf I}}\vert,$$ (23)

so the eigenvalues are the same as for A.

See also Brauer's Theorem, Condition Number, Eigenfunction, Eigenvector, Frobenius Theorem, Gersgorin Circle Theorem, Lyapunov's First Theorem, Lyapunov's Second Theorem, Ostrowski's Theorem, Perron's Theorem, Perron-Frobenius Theorem, Poincaré Separation Theorem, Random Matrix, Schur's Inequalities, Sturmian Separation Theorem, Sylvester's Inertia Law, Wielandt's Theorem

References

Arfken, G. ``Eigenvectors, Eigenvalues.'' §4.7 in Mathematical Methods for Physicists, 3rd ed. Orlando, FL: Academic Press, pp. 229-237, 1985.

Nash, J. C. ``The Algebraic Eigenvalue Problem.'' Ch. 9 in Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation, 2nd ed. Bristol, England: Adam Hilger, pp. 102-118, 1990.

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. ``Eigensystems.'' Ch. 11 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 449-489, 1992.