Hermitian maps and Hermitian matrices

We begin by considering a Hermitian matrix on ${\displaystyle \mathbb {C} ^{n}}$ (but the following discussion will be adaptable to the more restrictive case of symmetric matrices on ${\displaystyle \mathbb {R} ^{n}}$). We consider a Hermitian map A on a finite-dimensional complex inner product space V endowed with a positive definite sesquilinear inner product ${\displaystyle \langle \cdot ,\cdot \rangle }$. The Hermitian condition on ${\displaystyle A}$ means that for all x, y ∈ V,

$${\displaystyle \langle Ax,y\rangle =\langle x,Ay\rangle .}$$

An equivalent condition is that A^* = A, where A^* is the Hermitian conjugate of A. In the case that A is identified with a Hermitian matrix, the matrix of A^* is equal to its conjugate transpose. (If A is a real matrix, then this is equivalent to A^T = A, that is, A is a symmetric matrix.)

This condition implies that all eigenvalues of a Hermitian map are real: To see this, it is enough to apply it to the case when x = y is an eigenvector. (Recall that an eigenvector of a linear map A is a non-zero vector x such that Ax = λx for some scalar λ. The value λ is the corresponding eigenvalue. Moreover, the eigenvalues are roots of the characteristic polynomial.)

We provide a sketch of a proof for the case where the underlying field of scalars is the complex numbers.

By the fundamental theorem of algebra, applied to the characteristic polynomial of A, there is at least one eigenvalue λ₁ and eigenvector e₁, which must by definition be non-zero. Then since

$${\displaystyle \lambda _{1}\langle e_{1},e_{1}\rangle =\langle A(e_{1}),e_{1}\rangle =\langle e_{1},A(e_{1})\rangle ={\bar {\lambda }}_{1}\langle e_{1},e_{1}\rangle ,}$$

we find that λ₁ is real. Now consider the space K = span{e₁}^⊥, the orthogonal complement of e₁. By Hermiticity, K is an invariant subspace of A. Applying the same argument to K shows that A has an eigenvector e₂ ∈ K. Finite induction then finishes the proof.

The spectral theorem holds also for symmetric maps on finite-dimensional real inner product spaces, but the existence of an eigenvector does not follow immediately from the fundamental theorem of algebra. To prove this, consider A as a Hermitian matrix and use the fact that all eigenvalues of a Hermitian matrix are real.

The matrix representation of A in a basis of eigenvectors is diagonal, and by the construction the proof gives a basis of mutually orthogonal eigenvectors; by choosing them to be unit vectors one obtains an orthonormal basis of eigenvectors. A can be written as a linear combination of pairwise orthogonal projections, called its spectral decomposition. Let

$${\displaystyle V_{\lambda }=\{v\in V:Av=\lambda v\}}$$

be the eigenspace corresponding to an eigenvalue λ. Note that the definition does not depend on any choice of specific eigenvectors. V is the orthogonal direct sum of the spaces V_λ where the index ranges over eigenvalues.

In other words, if P_λ denotes the orthogonal projection onto V_λ, and λ₁, ..., λ_m are the eigenvalues of A, then the spectral decomposition may be written as

$${\displaystyle A=\lambda _{1}P_{\lambda _{1}}+\cdots +\lambda _{m}P_{\lambda _{m}}.}$$

If the spectral decomposition of A is ${\displaystyle A=\lambda _{1}P_{1}+\cdots +\lambda _{m}P_{m}}$, then ${\displaystyle A^{2}=(\lambda _{1})^{2}P_{1}+\cdots +(\lambda _{m})^{2}P_{m}}$ and ${\displaystyle \mu A=\mu \lambda _{1}P_{1}+\cdots +\mu \lambda _{m}P_{m}}$ for any scalar ${\displaystyle \mu .}$ It follows that for any polynomial f one has

$${\displaystyle f(A)=f(\lambda _{1})P_{1}+\cdots +f(\lambda _{m})P_{m}.}$$

The spectral decomposition is a special case of both the Schur decomposition and the singular value decomposition.

Normal matrices

The spectral theorem extends to a more general class of matrices. Let A be an operator on a finite-dimensional inner product space. A is said to be normal if A^*A = AA^*. One can show that A is normal if and only if it is unitarily diagonalizable. Proof: By the Schur decomposition, we can write any matrix as A = UTU^*, where U is unitary and T is upper-triangular. If A is normal, then one sees that TT^* = T^*T. Therefore, T must be diagonal since a normal upper triangular matrix is diagonal (see normal matrix). The converse is obvious.

In other words, A is normal if and only if there exists a unitary matrix U such that

$${\displaystyle A=UDU^{*},}$$

where D is a diagonal matrix. Then, the entries of the diagonal of D are the eigenvalues of A. The column vectors of U are the eigenvectors of A and they are orthonormal. Unlike the Hermitian case, the entries of D need not be real.

Possible absence of eigenvectors

The next generalization we consider is that of bounded self-adjoint operators on a Hilbert space. Such operators may have no eigenvalues: for instance let A be the operator of multiplication by t on ${\displaystyle L^{2}([0,1])}$, that is,^[3]

$${\displaystyle [A\varphi ](t)=t\varphi (t).}$$

This operator does not have any eigenvectors in ${\displaystyle L^{2}([0,1])}$, though it does have eigenvectors in a larger space. Namely the distribution ${\displaystyle \varphi (t)=\delta (t-t_{0})}$, where ${\displaystyle \delta }$ is the Dirac delta function, is an eigenvector when construed in an appropriate sense. The Dirac delta function is however not a function in the classical sense and does not lie in the Hilbert space L²[0, 1] or any other Banach space. Thus, the delta-functions are "generalized eigenvectors" of ${\displaystyle A}$ but not eigenvectors in the usual sense.

Spectral subspaces and projection-valued measures

In the absence of (true) eigenvectors, one can look for subspaces consisting of almost eigenvectors. In the above example, for example, where ${\displaystyle [A\varphi ](t)=t\varphi (t),\;}$ we might consider the subspace of functions supported on a small interval ${\displaystyle [a,a+\varepsilon ]}$ inside ${\displaystyle [0,1]}$. This space is invariant under ${\displaystyle A}$ and for any ${\displaystyle \varphi }$ in this subspace, ${\displaystyle A\varphi }$ is very close to ${\displaystyle a\varphi }$. In this approach to the spectral theorem, if ${\displaystyle A}$ is a bounded self-adjoint operator, then one looks for large families of such "spectral subspaces".^[4] Each subspace, in turn, is encoded by the associated projection operator, and the collection of all the subspaces is then represented by a projection-valued measure.

One formulation of the spectral theorem expresses the operator A as an integral of the coordinate function over the operator's spectrum ${\displaystyle \sigma (A)}$ with respect to a projection-valued measure.^[5]

$${\displaystyle A=\int _{\sigma (A)}\lambda \,dE_{\lambda }.}$$

When the self-adjoint operator in question is compact, this version of the spectral theorem reduces to something similar to the finite-dimensional spectral theorem above, except that the operator is expressed as a finite or countably infinite linear combination of projections, that is, the measure consists only of atoms.

Multiplication operator version

An alternative formulation of the spectral theorem says that every bounded self-adjoint operator is unitarily equivalent to a multiplication operator. The significance of this result is that multiplication operators are in many ways easy to understand.

Theorem.^[6] — Let A be a bounded self-adjoint operator on a Hilbert space H. Then there is a measure space (X, Σ, μ) and a real-valued essentially bounded measurable function f on X and a unitary operator U:H → L²(X, μ) such that

$${\displaystyle U^{*}TU=A,}$$

where T is the multiplication operator:

$${\displaystyle [T\varphi ](x)=f(x)\varphi (x),}$$

and ${\displaystyle \|T\|=\|f\|_{\infty }}$.

The spectral theorem is the beginning of the vast research area of functional analysis called operator theory; see also the spectral measure.

There is also an analogous spectral theorem for bounded normal operators on Hilbert spaces. The only difference in the conclusion is that now f may be complex-valued.

Direct integrals

There is also a formulation of the spectral theorem in terms of direct integrals. It is similar to the multiplication-operator formulation, but more canonical.

Let ${\displaystyle A}$ be a bounded self-adjoint operator and let ${\displaystyle \sigma (A)}$ be the spectrum of ${\displaystyle A}$. The direct-integral formulation of the spectral theorem associates two quantities to ${\displaystyle A}$. First, a measure ${\displaystyle \mu }$ on ${\displaystyle \sigma (A)}$, and second, a family of Hilbert spaces ${\displaystyle \{H_{\lambda }\},\,\,\lambda \in \sigma (A).}$ We then form the direct integral Hilbert space

$${\displaystyle \int _{\mathbf {R} }^{\oplus }H_{\lambda }\,d\mu (\lambda ).}$$

The elements of this space are functions (or "sections") ${\displaystyle s(\lambda ),\,\,\lambda \in \sigma (A),}$ such that ${\displaystyle s(\lambda )\in H_{\lambda }}$ for all ${\displaystyle \lambda }$. The direct-integral version of the spectral theorem may be expressed as follows:^[7]

Theorem — If ${\displaystyle A}$ is a bounded self-adjoint operator, then ${\displaystyle A}$ is unitarily equivalent to the "multiplication by ${\displaystyle \lambda }$" operator on

$${\displaystyle \int _{\mathbf {R} }^{\oplus }H_{\lambda }\,d\mu (\lambda )}$$

for some measure ${\displaystyle \mu }$ and some family ${\displaystyle \{H_{\lambda }\}}$ of Hilbert spaces. The measure ${\displaystyle \mu }$ is uniquely determined by ${\displaystyle A}$ up to measure-theoretic equivalence; that is, any two measure associated to the same ${\displaystyle A}$ have the same sets of measure zero. The dimensions of the Hilbert spaces ${\displaystyle H_{\lambda }}$ are uniquely determined by ${\displaystyle A}$ up to a set of ${\displaystyle \mu }$-measure zero.

The spaces ${\displaystyle H_{\lambda }}$ can be thought of as something like "eigenspaces" for ${\displaystyle A}$. Note, however, that unless the one-element set ${\displaystyle \lambda }$ has positive measure, the space ${\displaystyle H_{\lambda }}$ is not actually a subspace of the direct integral. Thus, the ${\displaystyle H_{\lambda }}$'s should be thought of as "generalized eigenspace"—that is, the elements of ${\displaystyle H_{\lambda }}$ are "eigenvectors" that do not actually belong to the Hilbert space.

Although both the multiplication-operator and direct integral formulations of the spectral theorem express a self-adjoint operator as unitarily equivalent to a multiplication operator, the direct integral approach is more canonical. First, the set over which the direct integral takes place (the spectrum of the operator) is canonical. Second, the function we are multiplying by is canonical in the direct-integral approach: Simply the function ${\displaystyle \lambda \mapsto \lambda }$.

Cyclic vectors and simple spectrum

A vector ${\displaystyle \varphi }$ is called a cyclic vector for ${\displaystyle A}$ if the vectors ${\displaystyle \varphi ,A\varphi ,A^{2}\varphi ,\ldots }$ span a dense subspace of the Hilbert space. Suppose ${\displaystyle A}$ is a bounded self-adjoint operator for which a cyclic vector exists. In that case, there is no distinction between the direct-integral and multiplication-operator formulations of the spectral theorem. Indeed, in that case, there is a measure ${\displaystyle \mu }$ on the spectrum ${\displaystyle \sigma (A)}$ of ${\displaystyle A}$ such that ${\displaystyle A}$ is unitarily equivalent to the "multiplication by ${\displaystyle \lambda }$" operator on ${\displaystyle L^{2}(\sigma (A),\mu )}$.^[8] This result represents ${\displaystyle A}$ simultaneously as a multiplication operator and as a direct integral, since ${\displaystyle L^{2}(\sigma (A),\mu )}$ is just a direct integral in which each Hilbert space ${\displaystyle H_{\lambda }}$ is just ${\displaystyle \mathbb {C} }$.

Not every bounded self-adjoint operator admits a cyclic vector; indeed, by the uniqueness in the direct integral decomposition, this can occur only when all the ${\displaystyle H_{\lambda }}$'s have dimension one. When this happens, we say that ${\displaystyle A}$ has "simple spectrum" in the sense of spectral multiplicity theory. That is, a bounded self-adjoint operator that admits a cyclic vector should be thought of as the infinite-dimensional generalization of a self-adjoint matrix with distinct eigenvalues (i.e., each eigenvalue has multiplicity one).

Although not every ${\displaystyle A}$ admits a cyclic vector, it is easy to see that we can decompose the Hilbert space as a direct sum of invariant subspaces on which ${\displaystyle A}$ has a cyclic vector. This observation is the key to the proofs of the multiplication-operator and direct-integral forms of the spectral theorem.

Functional calculus

One important application of the spectral theorem (in whatever form) is the idea of defining a functional calculus. That is, given a function ${\displaystyle f}$ defined on the spectrum of ${\displaystyle A}$, we wish to define an operator ${\displaystyle f(A)}$. If ${\displaystyle f}$ is simply a positive power, ${\displaystyle f(x)=x^{n}}$, then ${\displaystyle f(A)}$ is just the ${\displaystyle n}$-th power of ${\displaystyle A}$, ${\displaystyle A^{n}}$. The interesting cases are where ${\displaystyle f}$ is a nonpolynomial function such as a square root or an exponential. Either of the versions of the spectral theorem provides such a functional calculus.^[9] In the direct-integral version, for example, ${\displaystyle f(A)}$ acts as the "multiplication by ${\displaystyle f}$" operator in the direct integral:

$${\displaystyle [f(A)s](\lambda )=f(\lambda )s(\lambda ).}$$

That is to say, each space ${\displaystyle H_{\lambda }}$ in the direct integral is a (generalized) eigenspace for ${\displaystyle f(A)}$ with eigenvalue ${\displaystyle f(\lambda )}$.