Eigenvalue, eigenvector and eigenspace

From Wikipedia, the free encyclopedia

Fig. 1. In this shear mapping of the Mona Lisa, the picture was deformed in such a way that its central vertical axis (red vector) has not changed direction, but the diagonal vector (blue) has changed direction. Hence the red vector is an eigenvector of the transformation and the blue vector is not. Since the red vector was neither stretched nor compressed, its eigenvalue is 1. All vectors with the same vertical direction—i.e., parallel to this vector—are also eigenvectors, with the same eigenvalue. Together with the zero vector, they form the eigenspace for this eigenvalue.

In mathematics, eigenvalue, eigenvector, and eigenspace are related concepts in the field of linear algebra. The prefix eigen is the German word for innate, distinct, self. Linear algebra studies linear transformations, which are represented by matrices acting on vectors. Eigenvalues, eigenvectors and eigenspaces are properties of a matrix. They are computed by a method described below, give important information about the matrix, and can be used in matrix factorization. They have applications in areas of applied mathematics as diverse as finance and quantum mechanics.

In general, a matrix acts on a vector by changing both its magnitude and its direction. However, a matrix may act on certain vectors by changing only their magnitude, and leaving their direction unchanged (or possibly reversing it). These vectors are the eigenvectors of the matrix. A matrix acts on an eigenvector by multiplying its magnitude by a factor, which is positive if its direction is unchanged and negative if its direction is reversed. This factor is the eigenvalue associated with that eigenvector. An eigenspace is the set of all eigenvectors that have the same eigenvalue, together with the zero vector. The concepts cannot be formally defined without prerequisites, including an understanding of matrices, vectors, and linear transformations. The technical details are given below.

Mathematical definition

In linear algebra, there are two kinds of objects: scalars, which are just numbers; and vectors, which can be thought of as arrows, and which have both magnitude and direction (though more precisely a vector is a member of a vector space). In place of the ordinary functions of algebra, the most important functions in linear algebra are called "linear transformations", and a linear transformation is usually given by a "matrix", an array of numbers. Thus instead of writing f(x) we write M(v) where M is a matrix and v is a vector. The rules for using a matrix to transform a vector are given in the article linear algebra.

If the action of a matrix on a (nonzero) vector changes its magnitude but not its direction, then the vector is called an eigenvector of that matrix. A vector which is "flipped" to point in the opposite direction is also considered an eigenvector. Each eigenvector is, in effect, multiplied by a scalar, called the eigenvalue corresponding to that eigenvector. The eigenspace corresponding to one eigenvalue of a given matrix is the set of all eigenvectors of the matrix with that eigenvalue.

Many kinds of mathematical objects can be treated as vectors: ordered pairs, functions, harmonic modes, quantum states, and frequencies are examples. In these cases, the concept of direction loses its ordinary meaning, and is given an abstract definition. Even so, if this abstract direction is unchanged by a given linear transformation, the prefix "eigen" is used, as in eigenfunction, eigenmode, eigenface, eigenstate, and eigenfrequency.

Example

If a matrix is a diagonal matrix, then its eigenvalues are the numbers on the diagonal and its eigenvectors are basis vectors (of the standard basis) to which those numbers refer. For example, the matrix

stretches every vector to three times its original length in the x-direction and shrinks every vector to half its original length in the y-direction. Eigenvectors corresponding to the eigenvalue 3 are any multiple of the basis vector [1, 0]; together they constitute the eigenspace corresponding to the eigenvalue 3. Eigenvectors corresponding to the eigenvalue 0.5 are any multiple of the basis vector [0, 1]; together they constitute the eigenspace corresponding to the eigenvalue 0.5. In contrast, any other vector, [2, 8] for example, will change direction. The angle [2, 8] makes with the x-axis has tangent 4, but after being transformed, [2, 8] is changed to [6, 4], and the angle that vector makes with the x-axis has tangent 2/3.

History

Eigenvalues are often introduced in the context of linear algebra or matrix theory. Historically, however, they arose in the study of quadratic forms and differential equations.

Euler studied the rotational motion of a rigid body and discovered the importance of the principal axes. Lagrange realized that the principal axes are the eigenvectors of the inertia matrix.[1] In the early 19th century, Cauchy saw how their work could be used to classify the quadric surfaces, and generalized it to arbitrary dimensions.[2] Cauchy also coined the term racine caractéristique (characteristic root) for what is now called eigenvalue; his term survives in characteristic equation.[3]

Fourier used the work of Laplace and Lagrange to solve the heat equation by separation of variables in his famous 1822 book Théorie analytique de la chaleur.[4] Sturm developed Fourier's ideas further and brought them to the attention of Cauchy, who combined them with his own ideas and arrived at the fact that real symmetric matrices have real eigenvalues.[2] This was extended by Hermite in 1855 to what are now called Hermitian matrices.[3] Around the same time, Brioschi proved that the eigenvalues of orthogonal matrices lie on the unit circle,[2] and Clebsch found the corresponding result for skew-symmetric matrices.[3] Finally, Weierstrass clarified an important aspect in the stability theory started by Laplace by realizing that defective matrices can cause instability.[2]

In the meantime, Liouville studied eigenvalue problems similar to those of Sturm; the discipline that grew out of their work is now called Sturm-Liouville theory.[5] Schwarz studied the first eigenvalue of Laplace's equation on general domains towards the end of the 19th century, while Poincaré studied Poisson's equation a few years later.[6]

At the start of the 20th century, Hilbert studied the eigenvalues of integral operators by viewing the operators as infinite matrices.[7] He was the first to use the German word eigen to denote eigenvalues and eigenvectors in 1904, though he may have been following a related usage by Helmholtz. "Eigen" can be translated as "own", "peculiar to", "characteristic", or "individual"—emphasizing how important eigenvalues are to defining the unique nature of a specific transformation. For some time, the standard term in English was "proper value", but the more distinctive term "eigenvalue" is standard today.[8]

The first numerical algorithm for computing eigenvalues and eigenvectors appeared in 1929, when Von Mises published the power method. One of the most popular methods today, the QR algorithm, was proposed independently by John G.F. Francis[9] and Vera Kublanovskaya[10] in 1961.[11]

Technical definition

Linear transformations of a vector space, such as rotation, reflection, stretching, compression, shear or any combination of these, may be visualized by the effect they produce on vectors. In other words, they are vector functions. More formally, in a vector space L, a vector function A is defined if for each vector x of L there corresponds a unique vector y = A(x) of L. For the sake of brevity, the parentheses around the vector on which the transformation is acting are often omitted. A vector function A is linear if it has the following two properties:

• Additivity: A(x + y) = Ax + Ay
• Homogeneity: A(αx) = αAx

where x and y are any two vectors of the vector space L and α is any scalar.[12] Such a function is variously called a linear transformation, linear operator, or linear endomorphism on the space L.

Definition

Given a linear transformation A, a non-zero vector x is defined to be an eigenvector of the transformation if it satisfies the eigenvalue equation

for some scalar λ. In this situation, the scalar λ is called an eigenvalue of A corresponding to the eigenvector x.[13]

The key equation in this definition is the eigenvalue equation, Ax = λx. That is to say that the vector x has the property that its direction is not changed by the transformation A, but that it is only scaled by a factor of λ. Most vectors x will not satisfy such an equation: a typical vector x changes direction when acted on by A, so that Ax is not a multiple of x. This means that only certain special vectors x are eigenvectors, and only certain special scalars λ are eigenvalues. Of course, if A is a multiple of the identity matrix, then no vector changes direction, and all non-zero vectors are eigenvectors.