2 Linear Algebra

We include two primary references in this section: one is the textbook for Math 232 (Bretscher 2013) and the other is a standard textbook for advanced courses in linear algebra (Petersen 2012). We include the former for its familiarity and the latter for its abstractness and formalism. For example, results in Linear Algebra with Applications (Bretscher 2013) are stated over the real numbers and there is no discussion of Hermitian inner products; much of Linear Algebra (Petersen 2012) is done over an arbitrary field and spectral theory is given a robust treatment. For those interested in a compromise, featuring a pragmatic balance of theory with a special focus on materials of interest to physicists (e.g., separable Hilbert spaces, differential and integral operators, differential equations, etc.), we submit Introduction to Hilbert Spaces with Applications (Debnath and Mikusinski 2005). For another (decidedly abstract) treatment, see Abstract Algebra (Dummit and Foote 2003).

2.1 Basic Concepts

Definition 2.1 (Vector spaces) (Bretscher 2013, 167; Petersen 2012, 8) Let $F$ be a field. An $F$-vector space is a set $V$ with an operation $+: V \times V \to V$ and map $F \times V \to V$, denoted by \[ (x,y) \mapsto x+y \] and \[ (\alpha,x) \mapsto \alpha x, \] which are called vector addition and scalar multiplication, respectively, so that

$V,+$ is an abelian group.
Identity. $1 x = x$ for all $x \in V.$
Compatibility. $\alpha (\beta x) = (\alpha\beta) x$ for all $\alpha, \beta \in F,$ and $x \in V.$
Distributivity over scalars. $(\alpha+\beta)x = \alpha x + \beta x$ for all $\alpha, \beta \in F,$ and $x \in V.$
Distributivity over vectors. $\alpha(x + y) = \alpha x + \alpha y$ for all $\alpha \in F,$ and $x, y \in V.$

The identity vector with respect to addition is written $0 \in V.$

Definition 2.2 (Algebras) (Dummit and Foote 2003, 342) If an $F$-vector space $V$ is equipped with another binary operation $\cdot$ giving it a ring structure and also satisfying \[ \alpha(x \cdot y) = (\alpha x) \cdot y = x \cdot (\alpha y), \] for all $\alpha \in F$ and $x,y \in V,$ then $V$ is an $F$-algebra. If $\cdot$ is commutative then $V$ is a commutative $F$-algebra; otherwise, we might emphasize that $V$ is non-commutative. If every nonzero element in $V$ has a multiplicative inverse, we call it a division algebra.

Example 2.1 Every field $F$ is a commutative $F$-algebra; the trivial ring $0$ is also an $F$-algebra.

Example 2.2 $\mathbb{C}$ is a commutative $\mathbb{R}$-algebra with its usual field operations.

Example 2.3 The set of Hamilton quaternions $\mathbb{H}= \{ a + bi + cj + dk: a, b, c, d \in \mathbb{R}\}$ is a non-commutative $\mathbb{R}$-algebra (though, not a $\mathbb{C}$-algebra). Here multiplication is defined using the relations \[ i^2 = j^2 = k^2 = ijk = -1. \] Indeed, we will see that $\mathbb{H}$ is a division algebra.

Example 2.4 The set $\operatorname{Mat}_{m \times n}(F)$ of $m \times n$ matrices with entries in $F$ is an $F$-vector space with entrywise addition. The set of $n \times n$ matrices, here written more simply as $\operatorname{Mat}_n(F),$ is a non-commutative $F$-algebra with the additional operation of matrix multiplication.

Example 2.5 The ring of polynomials $F[x]$ is a commutative $F$-algebra.

Example 2.6 The set of all real-valued functions on $\mathbb{R},$ denoted $\operatorname{Func}(\mathbb{R},\mathbb{R}),$ is an $\mathbb{R}$-algebra when equipped with pointwise operations: \[ (f+g)(x) \coloneqq f(x)+g(x) \] and \[ (f \cdot g)(x) \coloneqq f(x)g(x). \] Similarly, one might consider functions on a subset $\Omega \subseteq \mathbb{R}^n$ or with values in $\mathbb{C}$; we write $\operatorname{Func}(\Omega,\mathbb{R})$ and $\operatorname{Func}(\Omega,\mathbb{C})$ for these algebras (over $\mathbb{R}$ and, for the latter, also over $\mathbb{C}$).

Example 2.7 If $F$ is a field, the set $F^n$ is an $F$-vector space with coordinate-wise addition. We follow the convention of writing vectors as columns: \[ x = \begin{pmatrix} \alpha_1 \\ \vdots \\ \alpha_n \end{pmatrix} \in F^n. \]

Definition 2.3 (Bretscher 2013, 170; Petersen 2012, 54) Let $V$ be an $F$-vector space. A subset $U \subseteq V$ is a subspace, denoted $U \leq V,$ if it is an $F$-vector space with the operations from $V.$ Equivalently, a non-empty subset $U$ is a subspace if, for every $x,y \in U$ and $\alpha, \beta \in F,$ we have $\alpha x+\beta y \in U.$ If $V$ is an $F$-algebra, a subspace $U$ is a subalgebra if it is also a subring.

Example 2.8 Every vector space contains the trivial subspace $\{0\}.$

Example 2.9 For $\Omega \subseteq \mathbb{R}^n,$ the following are all subalgebras of $\operatorname{Func}(\Omega,\mathbb{R}):$ \[ \begin{split} C^{}(\Omega,\mathbb{R}) & = \text{all continuous real-valued functions on } \Omega. \\ C^{k}(\Omega,\mathbb{R}) & = \text{all real-valued functions on } \Omega \text{ with continuous partial derivatives of order } k. \\ C^{\infty}(\Omega,\mathbb{R}) & = \text{all infinitely-differentiable real-valued functions on } \Omega. \\ \mathcal{P}(\Omega,\mathbb{R}) & = \text{functions on } \Omega \text{ that can be expressed as polynomials in $n$ variables.} \\ \end{split} \] We write $C^{}(\Omega,\mathbb{C}), C^{k}(\Omega,\mathbb{C}), C^{\infty}(\Omega,\mathbb{C}),$ and $\mathcal{P}(\Omega,\mathbb{C})$ for the analogous $\mathbb{C}$-valued subalgebras.

Proposition 2.1 (Petersen 2012, 56) Let $V$ be an $F$-vector space.

If $V_1, V_2 \leq V,$ then $V_1 \cap V_2 \leq V.$
If $V_1 \leq V$ and $V_2 \leq V_1,$ then $V_2 \leq V.$

Bases and Dimension

From this point on, all vector spaces are understood with respect to some fixed ground field $F$ unless stated otherwise.

Definition 2.4 (Complementary subspaces) (Petersen 2012, 57) Let $V$ be a vector space. We say that $V_1, V_2 \leq V$ are complementary if $V_1 \cap V_2 = \{0\}$ and every vector $x \in V$ can be expressed as $x = y+z$ for some $y \in V_1, z \in V_2.$

Proposition 2.2 Two subspaces $V_1, V_2 \leq V$ are complementary if and only if each $x \in V$ can be written uniquely as $x=y+z$ for $y \in V_1$ and $z \in V_2.$

Definition 2.5 (Bretscher 2013, 171; Petersen 2012, 56–57) Let $V$ be a vector space. Given $S \subseteq V,$ the span of $S$ is the smallest subspace of $V$ containing $S,$ i.e., the intersection of all subspaces containing $S.$ Equivalently, the span is the collection of all finite linear combinations in $S$: \[ \operatorname{Span}(S) \coloneqq \left\{ \sum_{i=1}^k \alpha_i x_i\ \middle|\ k \in \mathbb{N}, \alpha_i \in F, x_i \in S \right\}. \tag{2.1}\]

Definition 2.6 (Linear independence) (Bretscher 2013, 171; Petersen 2012, 72) Let $V$ be a vector space and $S \subseteq V$ non-empty. We say $S$ is linearly dependent if there are $x_1,\dots,x_n \in S$ and $\alpha_1,\dots,\alpha_n \in F$ not all zero with \[ \alpha_1 x_1 + \cdots + \alpha_n x_n = 0. \tag{2.2}\]

A set $\{x_1,\dots,x_n\}$ is instead said to be linearly independent if it is not linearly dependent; that is, the only solution to Equation 2.2 is given by taking all $\alpha_i = 0.$

Definition 2.7 (Bases) (Bretscher 2013, 172; Petersen 2012, 14) Let $V$ be a vector space. A collection of vectors $\mathscr{B} \subset V$ is said to be a basis for $V$ if $\mathscr{B}$ is linearly independent and \[ \operatorname{Span}(\mathscr{B}) = V. \] Often we refer to an ordered basis, which is a basis whose vectors are given a specific ordering, and we write $\mathscr{B}$ as a tuple rather than a set. When it will not cause confusion, we still use the notation $\mathscr{B} \subseteq V$ for ordered bases, understanding that $\mathscr{B}$ is a tuple rather than a set.

Theorem 2.1 (Dimension) (Bretscher 2013, 172; Petersen 2012, 15) Let $V$ be a vector space. All bases of $V$ have the same cardinality, called the dimension of $V$ and written $\dim_F V$ (or just $\dim V$ if the context is clear). If this cardinality is finite, we say that $V$ is finite-dimensional and write $\dim V < \infty$; otherwise, $V$ is infinite-dimensional.

Theorem 2.2 (Bretscher 2013, 172; Petersen 2012, 15) Let $V$ be a vector space with \[ S = \{x_1,\dots,x_n\} \subset V \] a linearly independent set. Then every element in $\operatorname{Span}(S)$ can be expressed uniquely as a linear combination of elements in $S.$ In particular, if $\dim V < \infty$ and $\mathscr{B}$ is a basis for $V,$ then every $x \in V$ can be expressed uniquely as a linear combination of basis elements.

Example 2.10 For $V = F^n,$ the standard basis is given by elements $e_i \in F^n$ with a $1$ in the $i$th entry and 0 elsewhere. The tuple $(e_1,\dots,e_n)$ is an ordered basis for $F^n,$ which is $n$-dimensional.

Example 2.11 The complexes $\mathbb{C}$ are 2-dimensional over $\mathbb{R},$ with $\{1,i\}$ as a basis.

Example 2.12 The quaternions $\mathbb{H}$ are 4-dimensional over $\mathbb{R},$ with $\{1,i,j,k\}$ as a basis.

Example 2.13 The matrix algebra $\operatorname{Mat}_n(F)$ is $n^2$-dimensional over $F.$

Example 2.14 The set $\{ x^n \mid n \in \mathbb{N}\},$ where we define $x^0 \coloneqq 1,$ is a basis for the polynomial algebra $F[x]$ over $F,$ which is therefore infinite-dimensional.

Example 2.15 The algebras $C(\mathbb{R}), C^k(\mathbb{R}),$ and $C^\infty(\mathbb{R})$ have infinite dimension over $\mathbb{R},$ each containing $\mathcal{P}(\Omega)$ as a subalgebra. Note that we know $\dim_{\mathbb{R}} \mathcal{P}(\Omega)$ is infinite by a (temporarily informal, cf. Definition 2.9) comparison to $\mathbb{R}[x],$ thinking of polynomials (the formal objects) as the functions defined in terms of polynomials.

Example 2.16 The trivial vector space (Example 2.8) is $0$-dimensional over $F$ with the basis $\varnothing.$

Decompositions

Definition 2.8 (Direct Sum) (Petersen 2012, 58) Given a pair of vector spaces $V$ and $W$ over the same field $F,$ we can make the Cartesian product $V \times W$ into a vector space by defining \[ (x,y) + (x',y') \coloneqq (x+x',y+y') \] and \[ \alpha (x,y) \coloneqq (\alpha x,\alpha y). \] To avoid such cumbersome notation, we think of elements in $V \times W$ as formal sums $x+y$ instead of pairs $(x,y),$ so the above rules become more aesthetically familiar: e.g., $\alpha(x+y) = \alpha x + \alpha y.$ This space is the direct sum of $V$ and $W$ and is denoted $V \oplus W.$

Remark 2.1. By construction, every $x \in V \oplus W$ has a unique decomposition $x = y+z$ for $y \in V$ and $z \in W.$ That is, $V$ and $W$ can be identified with natural complementary subspaces of $V \oplus W.$

Theorem 2.3 (Petersen 2012, 58) If $\{v_1,\dots,v_n\} \subset V$ and $\{w_1,\dots,w_m\} \subset W$ are bases, then \[ \{v_1,\dots,v_n,w_1,\dots,w_m\} \] is a basis for $V \oplus W.$ Hence $\dim (V \oplus W) = \dim V + \dim W.$

Theorem 2.4 (Existence of complements) (Petersen 2012, 61) Let $V$ be finite-dimensional with basis $\mathscr{B} = \{v_1,\dots,v_n\}$ and $U \leq V.$ Then it is possible to choose $\{v_{i_1},\dots,v_{i_k}\} \subseteq \mathscr{B}$ so that \[ V = U \oplus \operatorname{Span}\{v_{i_1},\dots,v_{i_k}\}. \]

Corollary 2.1 If $V$ is a vector space and $U \leq V,$ then $\dim U \leq \dim V.$ In particular, if $V$ is finite-dimensional, then $\dim U = \dim V$ if and only if $U = V.$

Linear Transformations

Definition 2.9 (Bretscher 2013, 178; Petersen 2012, 20) Let $V$ and $W$ be vector spaces over a field $F.$ Then a linear transformation (a.k.a. a linear map, a linear operator, or a homomorphism of $F$-vector spaces) is a function $L: V \to W$ satisfying \[ L(x+y)=L(x)+L(y) \] and \[ L(\alpha x) = \alpha L(x) \] for all $x,y \in V$ and $\alpha \in F.$ Equivalently, \[ L(\alpha x+\beta y) = \alpha L(x)+\beta L(y) \text{ for each } x,y \in V, \alpha,\beta \in F. \] We write $\operatorname{Hom}_F(V,W),$ or simply $\operatorname{Hom}(V,W)$ if the context is clear, for the set of all such maps. If $V=W,$ we abbreviate further to $\operatorname{End}(V) \coloneqq \operatorname{Hom}(V,V).$ If $L$ is a bijection, then its inverse is also linear; we call $L$ a linear isomorphism and write $V \cong W.$

Remark 2.2. Knowing the values of $L$ on a basis $\mathscr{B} \subset V$ determines $L$; we often define linear maps by deciding where to send basis elements, then extending linearly. In addition, we will often write $Lx$ instead of $L(x)$ to avoid cumbersome parentheses.

Example 2.17 The identity map $V \to V$ is a linear isomorphism. More generally, scaling by $\lambda \in F^\times$ is a linear isomorphism from $V \to V$ (known as a homothety) as is rotation about a particular axis through the origin.

Proposition 2.3 The composition of linear transformations is a linear transformation. Moreover, the composition of linear isomorphisms is a linear isomorphism.

Example 2.18 Complex conjugation is an $\mathbb{R}$-linear isomorphism $\mathbb{C}\to \mathbb{C}.$ Similarly, if $V$ is a field and $F$ is its canonical (a.k.a. prime (Dummit and Foote 2003, 511)) subfield then every element of $\operatorname{Aut}(V)$ is an $F$-linear isomorphism $V \to V$ (a.k.a., an $F$-equivariant automorphism).

Example 2.19 The map $\mathrm{D}: C^1([0,1]) \to C([0,1])$ given by $(\mathrm{D}f)(t) \coloneqq \frac{\mathop{}\!\mathrm{d}{f}}{\mathop{}\!\mathrm{d}{t}}$ is a linear map. Note that $\mathrm{D}$ is not injective, since every constant function maps to zero.

Example 2.20 The map $\mathrm{I}: C([0,1]) \to C^1([0,1])$ given by \[ (\mathrm{I}f)(t) = \int_0^t f(s) \mathop{}\!\mathrm{d}{x} \] is linear. By the fundamental theorem of calculus, $\mathrm{D}\circ \mathrm{I}= \mathbb{I}.$

Cosets and Quotients

Definition 2.10 (Bretscher 2013, 178; Petersen 2012, 64) If $L: V \to W$ is a linear map, then \[ \ker L \coloneqq \{ x \in V: Lx = 0 \} \] is called the kernel of $L$; the image of $L$ is defined as \[ \operatorname{im}L \coloneqq \{ y \in W \mid y = Lx \text{ for some } x \in V \}. \] Recall that $\ker L \leq V$ and $\operatorname{im}L \leq W$ and also that some books call $\ker L$ the nullspace. We call $\operatorname{Null}(L) \coloneqq \dim \ker L$ the nullity of $L$ and $\operatorname{rank}(L) \coloneqq \dim \operatorname{im}L$ the rank of $L.$

Definition 2.11 (Petersen 2012, 67) The map $V_1 \oplus V_2 \to V_1$ which forgets the $V_2$ term, i.e., \[ x+y \mapsto x, \] is linear and called a projection onto $V_1.$ Equivalently, a linear map $P: V \to V$ is called a projection if $P^2 = P,$ i.e., $Px = x$ for all $x \in \operatorname{im}P.$ In this setup, $\mathbb{I}-P: V \to V$ is also a projection such that $V_1 \coloneqq \operatorname{im}P = \ker(\mathbb{I}-P),$ $V_2 \coloneqq \ker P = \operatorname{im}(\mathbb{I}-P),$ and $V = V_1 \oplus V_2.$

Theorem 2.5 If $L: V \to W$ is a linear transformation and $Lx = y,$ then \[ L^{-1}(y) = x + \ker L \coloneqq \{ x + z \mid z \in \ker L \}. \] Informally: fibers of linear maps look the same. In particular, $L$ is one-to-one if and only if $\ker L$ is the trivial subspace. We call $x + \ker L$ the coset of $x$ with respect to $\ker L.$ This construction makes sense if we replace $\ker L$ with any subspace of $V,$ since any subspace can be realized as the kernel of some linear map.

Example 2.21 Given a differential equation of the form \[ \mathrm{D}f = g, \] where $\mathrm{D}$ is some linear differential operator and $g$ is a given function, a common technique is to first find all solutions to the equation $\mathrm{D}f = 0$ (i.e., $\ker D$) and then find a particular function $f_p$ so that $\mathrm{D}f_p = g.$ Then the space of all solutions is the coset $f_p + \ker \mathrm{D}.$

To see this in action, consider the differential equation \[ f''(t)+f(t)= - \sin t. \] By the arduous process of educated guesses, one can find a particular solution $f_p(t) = \tfrac{t}{2} \cos t.$ Moreover, by considering the associated characteristic equation, one can show that all real-valued solutions to $f''(t)+f(t)=0$ are of the form $a \cos t + b \sin t$ for $a,b \in \mathbb{R}.$ Hence \[ \left\{ \tfrac{t}{2} \cos t + a \cos t + b \sin t \middle| a, b \in \mathbb{R}\right\} \] is the set of all solutions to $f''(t) + f(t) = -\sin t.$ See, e.g., (Logan 2015, 100) for further reading.

Remark 2.3. Recall the infamous $+C$ that calculus instructors are so fond of. They were generously reminding you that solutions to differential equations are really cosets!

Definition 2.12 (Quotient vector spaces) (Dummit and Foote 2003, 108; Petersen 2012, 108) Let $V$ be an $F$-vector space with $U \leq V.$ Then the set of cosets \[ V/U \coloneqq \{ x + U \mid x \in V \} \] is a (well-defined) $F$-vector space with the operations \[ (x+U) + (y+U) \coloneqq (x+y) + U \] and \[ \alpha (x+U) \coloneqq (\alpha x)+U. \] We call $V/U$ the quotient of $V$ with respect to $U$ and $\dim V/U$ the codimension of $U \leq V.$ If we suppose further that $V$ is finite-dimensional, then we have \[ \dim V/U = \dim V - \dim U, \] i.e., the codimension of $U$ is the same as the dimension of any complementary subspace to $U.$

Proof (Sketch). If $\{v_1,\dots,v_n\}$ is a basis for $V$ and $V = U \oplus \operatorname{Span}\{v_{i_1},\dots,v_{i_k}\},$ then show \[ \{ v_{i_1} + U, \dots, v_{i_k} + U \} \] is a basis for $V/U.$

Theorem 2.6 (Noether’s isomorphism theorem) (Dummit and Foote 2003, 412; Petersen 2012, 109) Let $L: V \to W$ be a linear map. Then there is a natural isomorphism \[ V/ \ker L \cong \operatorname{im}L. \] Moreover, there are inclusion-respecting bijections: \[ \{ \text{subspaces of $V$ containing $\ker L$} \} \leftrightarrow \{ \text{subspaces of $V/\ker L$} \} \leftrightarrow \{ \text{subspaces of $\operatorname{im}L$} \}. \]

Corollary 2.2 (Rank-nullity theorem) If $V$ is finite-dimensional and $L: V \to W$ is linear, then \[ \dim L = \operatorname{Null}L + \operatorname{rank}L. \]

Corollary 2.3 (Isomorphism criteria) Suppose $V$ and $W$ are vector spaces of the same finite dimension and that $L: V \to W$ is a linear transformation. Then the following are equivalent:

$L$ is an isomorphism.
$\ker L = \{0\}.$
$\operatorname{im}L = W.$
$L$ sends a basis of $V$ to a basis of $W.$

Corollary 2.4 Two finite dimensional vector spaces over the same field are isomorphic if and only if they have the same dimension.

Matrices

Definition 2.13 (Bretscher 2013, 187; Petersen 2012, 48) Given a linear transformation $L: V \to W$ and a pair of ordered bases $\mathscr{A} = (v_1,\dots,v_n)$ and $\mathscr{B} = (w_1,\dots,w_m)$ for $V$ and $W,$ respectively, we associate a matrix (i.e., an $m \times n$ array of scalars) to $L$ as follows. For each basis element $v_j \in V,$ we know the image $Lv_j \in W = \operatorname{Span}(\mathscr{B})$ can be expressed uniquely as a linear combination: \[ Lv_j = \ell_{1,j} w_1 + \ell_{2,j} w_2 + \cdots + \ell_{m,j} w_m, \] with each $\ell_{i,j} \in F.$ For bookkeeping, we arrange these coefficients into columns: \[ {}_{\mathscr{B}}[L]_{\mathscr{A}} \coloneqq \begin{pmatrix} \ell_{1,1} & \cdots & \ell_{1,n} \\ \vdots & \ddots & \vdots \\ \ell_{m,1} & \cdots & \ell_{m,n} \\ \end{pmatrix} \in \operatorname{Mat}_{m \times n}(F). \] When the bases are clear, we might simply write $M_L$ for this matrix.

For an arbitrary $x \in V$ expressed in the basis $\mathscr{A},$ i.e., $x = \alpha_1 v_1 + \cdots + \alpha_n v_n,$ we can compute \[ Lx = (\ell_{1,1} \alpha_1 + \cdots + \ell_{1,n} \alpha_n) w_1 + \cdots + (\ell_{m,1} \alpha_1 + \cdots + \ell_{m,n} \alpha_n) w_m. \] More compactly, writing $(Lx)_i$ for the $w_i$ coefficient of $Lx,$ we have \[ (Lx)_i = \sum_{j=1}^n \ell_{i,j} \alpha_j. \] If we write a column vector $[x]_\mathscr{A}$ with the coefficient of $v_j$ in the $j$th entry, then we can compute $L(x)$ using the standard definition of matrices acting on column vectors: \[ [L(x)]_\mathscr{B} = {}_{\mathscr{B}}[L]_{\mathscr{A}} \; [x]_\mathscr{A}. \] Hence, given bases, much of the theory of linear maps can be reduced to that of matrices from $\operatorname{Mat}_{m \times n}(F)$ acting on column vectors from $F^n$ (cf. Example 2.7). For example, the rank of a linear map is equal to the rank of its associated matrix, which can be computed using the concrete techniques of row reduction; see (Bretscher 2013, 26; Petersen 2012, 82).

We will most often be interested in maps $L: V \to V$ with the same basis $\mathscr{B}$ for the domain and the codomain, where we simply write $[L]_{\mathscr{B}}$ for the associated matrix.

Theorem 2.7 Composition of linear transformations corresponds to matrix multiplication. That is, if $T: U \to V$ and $L: V \to W$ are linear maps with $\mathscr{A}, \mathscr{B},$ and $\mathscr{C}$ ordered bases for the vector spaces $U, V,$ and $W,$ respectively, then \[ {}_{\mathscr{C}}[L \circ T]_{\mathscr{A}} = {}_{\mathscr{C}}[L]_{\mathscr{B}} \; {}_{\mathscr{B}}[T]_{\mathscr{A}}. \]

Example 2.22 Consider the operator $\mathrm{D}$ from Example 2.19 restricted to polynomials of degree less than $n+1.$ Take the ordered basis $(1, x, x^2, \dots, x^n).$ Then $\mathrm{D}$ is associated to the matrix \[ \begin{pmatrix} 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 2 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & n \\ 0 & 0 & 0 & \cdots & 0 \end{pmatrix}. \] We can compute $\frac{\mathop{}\!\mathrm{d}{}}{\mathop{}\!\mathrm{d}{x}} (x^4-3x^2+7x+2) = 4x^3-6x+7$ by way of matrix multiplication: \[ \begin{pmatrix} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 2 & 0 & 0 \\ 0 & 0 & 0 & 3 & 0 \\ 0 & 0 & 0 & 0 & 4 \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix} \begin{pmatrix} 2 \\ 7 \\ -3 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 7 \\ -6 \\ 0 \\ 4 \\ 0 \end{pmatrix}. \]

Theorem 2.8 (Bretscher 2013, 194; Petersen 2012, 48) Let $V$ be finite-dimensional with $L: V \to V$ a linear map and consider two ordered bases $\mathscr{A}$ and $\mathscr{B} = (v_1,\dots,v_n).$ Then \[ S \; [L]_{\mathscr{B}} = [L]_{\mathscr{A}} \; S \] and \[ [L]_{\mathscr{B}} = S^{-1} \; [L]_{\mathscr{A}} \; S, \] where $S$ is the invertible matrix whose $i$th column is given by $v_i$ expressed in the basis $\mathscr{A}.$ The latter is known as conjugation by the matrix $S.$

Example 2.23 Consider complex conjugation, as in Example 2.18. The change of basis matrix from $\mathscr{A} = \{1+i, 1-i\}$ to $\mathscr{B} = \{1,i\}$ is given by \[ S = \begin{pmatrix} 1/2 & 1/2 \\ 1/2 & -1/2 \\ \end{pmatrix}, \] since $\tfrac{1}{2} (1+i) + \tfrac{1}{2} (1-i) = 1$ and $\tfrac{1}{2} (1+i) + \tfrac{-1}{2} (1-i) = i.$ Noting that $S^{-1} = 2 S,$ we have \[ \begin{pmatrix} 1 & 0 \\ 0 & -1 \\ \end{pmatrix} = S^{-1} \begin{pmatrix} 0 & 1 \\ 1 & 0 \\ \end{pmatrix} S. \] This is an example of diagonalizing a matrix, i.e., changing from a less convenient basis into a more convenient basis, to be reviewed in Section 2.3.2.

Definition 2.14 If $V$ and $W$ are finite-dimensional and $L: V \to V$ and $T: W \to W$ are a pair of linear maps, there is an induced map $L \oplus T: V \oplus W \to V \oplus W$ given by \[ (L \oplus T)(x+y) \coloneqq L(x) + T(y) \] for each $x \in V$ and $y \in W.$ Given ordered bases for $V$ and $W,$ so that $L$ and $T$ are expressed by the matrices $M_L$ and $M_T,$ respectively, the map $L \oplus T$ is represented by the block matrix \[ \begin{pmatrix} M_L & 0 \\ 0 & M_T \end{pmatrix}. \] Block matrices are especially important because they visually depict how the associated linear map respects (or fails to respect) a subspace decomposition.

Example 2.24 In the previous sense, a diagonal matrix \[ \begin{pmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{pmatrix} \] can be understood as the direct sum of many scaling transformations: $\lambda_1 \oplus \lambda_2 \oplus \cdots \oplus \lambda_n,$ which each $\lambda_i$ acts on the subspace spanned by $e_i.$ In this sense, diagonalized matrices are nice because they have been decomposed into many $1$-dimensional linear transformations (a.k.a. scaling) that do not interact with one another.

Definition 2.15 There are many equivalent and useful definitions for the determinant of $A \in \operatorname{Mat}_n(F),$ written $\det(A) \in F.$ In particular, the determinant is the only continuous map $\phi: \operatorname{Mat}_n(\mathbb{R}) \to \mathbb{R}$ satisfying $\phi(\lambda \mathbb{I}) = \lambda^n$ and \[ \phi(AB) = \phi(A) \phi(B) \] for all $A, B \in \operatorname{Mat}_n(\mathbb{R})$ and $\lambda \in \mathbb{R}$; there are similar characterizing properties over arbitrary fields. From this one can show, among other things, that $\det(A) = 0$ if and only if $A$ is not invertible and $\det(A^{-1}) = \det(A)^{-1}$ for all invertible matrices.

Another, more geometric way of understanding determinants over $F = \mathbb{R}$ is in terms of the volume of the paralellepiped determined by the columns of the matrix. In other words, the determinant measures how a matrix dilates or contracts unit volume. One can use this formulation to again see that $\det(A) = 0$ if and only if $\{0\} \subset \ker A,$ since a matrix with non-trivial kernel must collapse at least one axis in $\mathbb{R}^n$ to zero.

Determinants can be computed in a number of ways, in particular the recursive Laplace expansion in terms of a weighted sum of minors (a.k.a. submatrices obtained by deleting one row and one column) or via adjugates. Lastly, for any finite-dimensional vector space $V,$ we can speak of the determinant of linear maps $L: V \to V$ by fixing an ordered basis $\mathscr{A},$ and computing $\det([L]_{\mathscr{A}}).$ This is well-defined, since choosing a different basis $\mathscr{B}$ for $V$ amounts to conjugation and therefore \[ \det([L]_{\mathscr{B}}) = \det(S^{-1} \; [L]_{\mathscr{A}} \; S) = \det(S)^{-1} \det( [L]_{\mathscr{A}}) \det(S) = \det( [L]_{\mathscr{A}}). \]

Definition 2.16 (Bretscher 2013, 194; Petersen 2012, 48) The trace of a matrix $A \in \operatorname{Mat}_n(F)$ is given by the sum of its diagonal entries, i.e., writing $a_{i,j}$ for the entry of $A$ in the $i$th row and $j$th column, $\operatorname{Tr}(A) \coloneqq a_{1,1} + \cdots + a_{n,n}.$ The trace satisfies a cyclic invariance property: \[ \operatorname{Tr}(AB) = \operatorname{Tr}(BA) \] for all $A, B \in \operatorname{Mat}_n(F),$ as well as linearity. Moreover, the trace is the unique linear map $\operatorname{Mat}_n(F) \to F$ satisfying the cyclic invariance property and $\operatorname{Tr}(\mathbb{I}) = n.$ The trace of a linear map on a finite-dimensional vector space $V$ is defined by fixing a basis and computing the trace of the associated matrix; as with determinants, this is independent of the basis choice.

Definition 2.17 The commutator of two linear operators $A, B: V \to V$ is defined as \[ [A,B] \coloneqq AB - BA. \] This expression is so named because $AB = BA$ if and only if $[A,B] = 0.$

Definition 2.18 The operator $T: C^\infty([0,1]) \to C^\infty([0,1])$ given by $(Tf)(t) = t\ f(t)$ is linear. This map does not commute with $\mathrm{D}= \frac{\mathop{}\!\mathrm{d}{}}{\mathop{}\!\mathrm{d}{t}},$ since \[ (\mathrm{D}T f)(t) = f(t) + t f'(t) \not = t f'(t) = (T \mathrm{D}f)(t). \] Indeed, $[\mathrm{D},T] = \mathbb{I}.$ Physicists like to make a big deal about this sort of thing.

2.2 Hermitian Inner Products

In this and subsequent sections we only consider vector spaces over $F = \mathbb{R}$ or $F = \mathbb{C}.$

Definition 2.19 (Inner product) (Bretscher 2013, 249; Petersen 2012, 209) An inner product on any real vector space $V$ is a map $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}$ satisfying

Symmetry. $\langle x, y \rangle = \langle y, x \rangle.$
Linearity in the second argument. $\langle x, \alpha y+\beta z \rangle = \alpha \langle x, y \rangle + \beta \langle x, z \rangle.$
Positive-definiteness. $\langle x, x \rangle > 0$ for all $x \not = 0.$

Note that combining the first two axioms also gives linearity in the first argument.

Remark 2.4. For a vector space $V$ defined over the complex numbers, we cannot hope for these axioms to hold. For example, we would need to have \[ 0 < \langle ix, ix \rangle = i^2 \langle x, x \rangle = - \langle x, x \rangle < 0, \] which is a contradiction. However, complex conjugation provides a suitable modification: we know $z \overline{z} = |z|^2 \geq 0$ for all $z \in \mathbb{C},$ with equality if and only if $z=0.$

Definition 2.20 (Hermitian inner product) (Bretscher 2013, 213; Debnath and Mikusinski 2005, 94) A Hermitian inner product on a complex vector space $V$ is any map $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{C}$ satisfying

Skew-symmetry, a.k.a, conjugate symmetry. $\langle x, y \rangle = \overline{\langle y, x \rangle}.$
Linearity in the second argument. $\langle x, \alpha y+\beta z \rangle = \alpha \langle x, y \rangle + \beta \langle x, z \rangle.$
Positive-definiteness. $\langle x, x \rangle > 0$ for all $x \not = 0.$

Combining the first two axioms, we see that this pairing is antilinear in the first argument: \[ \langle \alpha x + \beta y, z \rangle = \overline{\alpha} \langle x, z \rangle + \overline{\beta} \langle y, z \rangle. \] Some books take linearity in the first argument and hence antilinearity in the second—this is a (divisive) matter of convention.

Definition 2.21 A vector space $V$ over $\mathbb{R}$ (or $\mathbb{C}$) equipped with an inner product (respectively, a Hermitian inner product) is an inner product space. When emphasis is needed, we might refer to $V$ as a real (respectively, complex) inner product space.

The notion of inner product generalizes the notion of dot product, which geometrically corresponds to the cosine of the angle between vectors. Indeed, inner product spaces have a natural notion of distance as well as angles.

Proposition 2.4 Let $V$ be an inner product space and fix $x, x' \in V$. If \[ \langle x, y \rangle = \langle x', y \rangle \] hold for all $y \in V$, then $x = x'$.

Proof. We can rewrite the above as $\langle x-x', y \rangle$; choosing $y = x-x'$ gives $\langle x-x', x-x' \rangle = 0$, from which positive-definiteness implies $x-x'=0$.

Proposition 2.5 (Bretscher 2013, 218; Debnath and Mikusinski 2005, 97) Every inner product space has a natural norm $V \to \mathbb{R}^{\geq 0}$ given by \[ \|x\| \coloneqq \sqrt{\langle x, x \rangle} \] which satisfies

Absolute homogeneity. $\|ax\| = |a| \, \|x\|.$
Triangle inequality. $\|x+y\| \leq \|x\| + \|y\|.$
Pythagorean theorem. $\|x+y\|^2 = \|x\|^2 + \|y\|^2 \text{ if } \langle x, y \rangle = 0.$
Cauchy–Schwarz inequality. $\langle x, y \rangle \leq \|x\| \, \|y\|.$
Parallelogram law. $\|x+y\|^2 + \|x-y\|^2 = 2\|x\|^2 + 2\|y\|^2.$

Definition 2.22 If $V$ is an inner product space, $x, y \in V$ are called orthogonal if $\langle x, y \rangle = 0.$ We say that $x \in V$ is a unit vector if $\|x\| = 1$ and we call the process of replacing a nonzero vector $x \in V$ with the unit vector $\tfrac{x}{\|x\|}$ normalizing.

Example 2.25 The vector space $\mathbb{R}^n$ is an inner product space over $\mathbb{R}$ with the usual notion of dot product: $\langle x, y \rangle \coloneqq x^\top y,$ where the superscript ${}^\top$ denotes the transpose. Similarly, $\mathbb{C}^n$ is an inner product space over $\mathbb{C}$ with the Hermitian dot product: $\langle x, y \rangle \coloneqq x^* y = \overline{x}^\top y,$ i.e., \[ \left\langle \begin{pmatrix} \alpha_1 \\ \vdots \\ \alpha_n \end{pmatrix}, \begin{pmatrix} \beta_1 \\ \vdots \\ \beta_n \end{pmatrix} \right\rangle \coloneqq \overline{\alpha_1} \beta_1 + \cdots + \overline{\alpha_n} \beta_n. \]

Orthogonal Complements

Proposition 2.6 (Bretscher 2013, 236; Debnath and Mikusinski 2005, 127) If $V$ is an inner product space and $U \leq V,$ then \[ U^\perp \coloneqq \{ x \in V: \langle x, y \rangle = 0 \text{ for all } y \in U \} \] is a subspace of $V$ called the orthogonal complement of $U.$ This name is deserved because $V = U \oplus U^\perp,$ i.e., every $x \in V$ can be written uniquely as $x = y+z$ for $y \in U$ and $z \in U^\perp.$

Remark 2.5. If $V$ is an inner product space and $\{u_1,\dots, u_k\}$ is a basis for a subspace $U \leq V$, then the kernel of the operator \[ \begin{aligned} T: V & \to \mathbb{C}^k \\ x & \mapsto ( \langle u_1, x \rangle, \cdots, \langle u_k, x \rangle ) \end{aligned} \] is exactly the orthogonal complement $U^\perp$.

In particular, writing $T$ in terms of a basis for $V$ and computing the row reduced echelon form gives a basis for $U^\perp$. Under the standard Hermitian inner product on $V = \mathbb{C}^n$, this is simply the nullspace of the matrix whose rows are the complex conjugate of the vectors $u_1, \dots, u_k$; i.e., if $L = [u_1 \ \cdots \ u_k]$, then \[ U^\perp = \ker(L^*). \]

Proposition 2.7 If $V$ is a finite-dimensional inner product space and $U \leq V,$ then \[ (U^\perp)^\perp = U. \]

Theorem 2.9 (Orthogonal projection) (Bretscher 2013, 213; Debnath and Mikusinski 2005, 130) Let $V$ be a finite-dimensional inner product space with $U \leq V.$ Then there is a unique projection $P: V \to V$ with $\operatorname{im}P = U$ and $\ker P = U^\perp.$ We call $P$ the orthogonal projection onto $U.$ In particular, $y = Px$ is the unique vector in $U$ such that $\langle x - y, z \rangle = 0$ for all $z \in W$.

Remark 2.6. In general, we say that a projection $P$ is orthogonal if \[ \langle Px, y \rangle = \langle x, Py \rangle, \] for all $x,y \in V,$ i.e., if $P$ is self-adjoint (see Section 2.2.3).

Example 2.26 The matrix $P = \begin{psmallmatrix} 0 & 0 \\ a & 1 \end{psmallmatrix}$ is a projection onto the second coordinate in $\mathbb{R}^2$ and is orthogonal with respect to the usual dot product if and only if $a=0.$

Example 2.27 If $V$ is an inner product space and $x_0 \in V$ is a unit vector, then the linear map $P: V \to V$ given by $Px \coloneqq \langle x_0, x \rangle x_0$ is an orthogonal projection onto $U = \operatorname{Span}\{x_0\}.$

Orthonormal Bases

Definition 2.23 Let $V$ be a real or complex inner product space. A collection of vectors $\{v_1,\dots,v_n\} \subset V$ is called pairwise orthonormal if \[ \langle v_i, v_j \rangle = \left\{ \begin{array}{ll} 1 & \text{ if } i = j\\ 0 & \text{ otherwise. } \end{array} \right. \] If $\mathscr{B} \subset V$ is a basis of pairwise orthonormal vectors, $\mathscr{B}$ is an orthonormal basis for $V.$

Example 2.28 The standard bases of $\mathbb{R}^n$ and $\mathbb{C}^n$ (see Example 2.10 and Example 2.25) are orthonormal bases with respect to the usual dot and Hermitian dot products, respectively.

Example 2.29 ($L^2$ inner products) We can give $C^{}([0,2\pi],\mathbb{R})$ an inner product via \[ \langle f, g \rangle \coloneqq \frac{1}{\pi} \int_0^{2 \pi} f(t) g(t) \mathop{}\!\mathrm{d}{t}. \tag{2.3}\]

Then the infinite set $\{ \tfrac{1}{\sqrt{2}}, \sin t, \cos t, \sin(2t), \cos(2t), \sin(3t), \dots \}$ is pairwise orthonormal. If we instead consider the space $C^{}([-\pi,\pi],\mathbb{C})$ with the inner product \[ \langle f, g \rangle \coloneqq \int_{-\pi}^{\pi} \overline{f(t)} g(t) \mathop{}\!\mathrm{d}{t}, \] then the set $\left\{ \tfrac{1}{\sqrt{2\pi}}e^{int} \middle| n \in \mathbb{Z}\right\}$ is pairwise orthonormal.

Remark 2.7. Orthonormal bases are convenient in many ways. In particular, to express an arbitrary vector $x \in V$ in terms of an orthonormal basis $\{v_1, \dots, v_n \} \subset V,$ one can simply apply inner products. In particular, we have \[ x = \alpha_1 v_1 + \cdots + \alpha_n v_n \] where \[ \alpha_i \coloneqq \langle v_i, x \rangle. \] This gives an alternative form of the Pythagorean theorem: \[ \|x\|^2 = \langle x, x \rangle = \sqrt{|\alpha_1|^2+\cdots+|\alpha_n|^2}. \] Moreover, if $y = \beta_1 v_1 + \cdots + \beta_n v_n \in V$ is another vector, then we have \[ \langle x, y \rangle = \overline{\alpha_1} \beta_1 + \cdots + \overline{\alpha_n} \beta_n. \] In other words, orthonormal bases reduce abstract inner products to dot products.

Theorem 2.10 (Gram–Schmidt process) (Bretscher 2013, 218; Petersen 2012, 225) Given a basis $\mathscr{B} \subset V$ for $V$ an inner product space, one can always obtain an orthonormal basis via a process of iterated projections. To be specific, given $\{v_1,\dots,v_n\}$ we produce $\{u_1,\dots,u_n\}$ via \[ \begin{gathered} u_1 = \frac{v_1}{\|v_1\|}, \\ u_2 = \frac{v_2 - \langle u_1,v_2 \rangle u_1}{\|v_2 - \langle u_1,v_2 \rangle u_1\|}, \\ u_3 = \frac{v_3 - \langle u_1,v_3 \rangle u_1 - \langle u_2,v_3 \rangle u_2}{\|v_3 - \langle u_1,v_3 \rangle u_1 - \langle u_2,v_3 \rangle u_2\|}, \end{gathered} \] and so on.

Functionals and Adjoints

Definition 2.24 If $V$ is a vector space over $F,$ a functional is a linear map $V \to F.$

Example 2.30 The maps $f(t) \mapsto \int_0^1 f(t) \mathop{}\!\mathrm{d}{t}$ and $f(t) \mapsto f(0)$ are functionals on $C([0,1]).$

Proposition 2.8 Given functionals $f, h: V \to F$ and $\alpha, \beta \in F,$ we define \[ \begin{aligned} \alpha f + \beta h : V & \to F \\ x & \mapsto \alpha f(x) + \beta h(x). \end{aligned} \] This new mapping is also a functional. Moreover, these notions of addition and scaling show that the set $V^*$ of all functionals on $V$ is itself a vector space (called the dual space of $V$).

Proposition 2.9 If $\dim V < \infty,$ then $\dim V = \dim V^*.$ In particular, if $\{v_1,\dots,v_n\} \subset V$ is a basis, there is a canonical dual basis $\{f_1,\dots,f_n\} \subset V^*,$ where $f_i(v_i) = 1$ and $f_i(v_j) = 0$ for all $i \not = j.$

Lemma 2.1 If $f: V \to F$ is a non-trivial functional on a finite-dimensional vector space $V,$ then $\operatorname{rank}(f) = 1.$ If $V$ is an inner product space, then $(\ker f)^\perp \subset V$ is $1$-dimensional.

Theorem 2.11 (Riesz representation) (Debnath and Mikusinski 2005, 133) If $V$ is a finite-dimensional inner product space and $f: V \to F$ is a functional, there is a unique $x_f \in V$ such that \[ f(x) = \langle x_f, x \rangle \] for all $x \in V.$ In other words, an inner product gives a canonical correspondence \[ \begin{aligned} \varphi: V & \to V^* \\ y & \mapsto \varphi(y) = (x \mapsto \langle y, x \rangle) \end{aligned} \] that is an isomorphism if $F = \mathbb{R}$ and an anti-isomorphism if $F = \mathbb{C}$, i.e., \[ \varphi(\alpha y) = \overline{\alpha} \varphi(y). \]

Proof (Sketch). If $f: V \to F$ is non-trivial, fix a unit vector $y \in (\ker f)^\perp.$ The the assignment $x \mapsto \langle y, x \rangle y$ is an orthogonal projection onto $(\ker f)^\perp,$ so we can write (c.f. Example 2.27): \[ x = \underbrace{x - \langle y, x \rangle y}_{\in \ker f} + \underbrace{\langle y, x \rangle y}_{\in (\ker f)^\perp}. \] But this means that \[ f(x) - \langle y, x \rangle f(y) = 0 \] for all $x \in V,$ i.e., taking $x_f = \overline{f(y)} y$ gives the desired result.

Definition 2.25 If $V$ and $W$ are both $F$-vector spaces, a map $\mu: V \times V \to W$ is said to be a sesquilinear if, for all $\alpha, \beta \in F$ and $x,y,z \in V,$ we have: \[ \mu(\alpha x+\beta y,z) = \overline{\alpha} \mu(x,z) + \overline{\beta} \mu(y,z) \] and \[ \mu(x,\alpha y+\beta z) = \alpha \mu(x,y) + \beta \mu(x,z). \] If $W = F,$ $\mu$ is a sesquilinear pairing. In the $F = \mathbb{R}$ context, $\mu$ is simply called bilinear.

Example 2.31 An inner product on $V$ is a sesquilinear pairing. If $A$ and $B$ are linear operators on $V,$ then $(x,y) \mapsto \langle Ax, By \rangle$ is a sesquilinear pairing. Similarly, if $f, h: V \to F$ are linear functionals, then $(x,y) \mapsto \overline{f(x)} h(y)$ is sesquilinear.

Theorem 2.12 If $\mu$ is a sesquilinear pairing on a a finite-dimensional inner product space $V,$ then there is a unique linear operator $A_\mu: V \to V$ such that, for all $x, y \in V,$ we have \[ \mu(x,y) = \langle A_\mu x, y \rangle, \] i.e., any sesquilinear pairing can be given in terms of an operator and the inner product.

Proof (Sketch). For a fixed choice of $x \in V,$ the map $y \mapsto \mu(x,y)$ is a linear functional. Then, by Riesz representation (Theorem 2.11), there is a unique vector $x_\mu \in V$ such that $\mu(x,y) = \langle x_\mu, y \rangle$ for all $y \in V.$ We need to show that the assignment $x \mapsto x_\mu$ defines a linear operator $A_\mu \mid V \to V.$ Indeed, for any $\alpha, \beta \in F$ and $x, x', y \in V$ we have \[ \begin{split} \langle A_\mu (\alpha x + \beta x'), y \rangle & = \mu(\alpha x + \beta x',y) \\ & = \overline{\alpha} \mu(x,y) + \overline{\beta} \mu(x',y) \\ & = \overline{\alpha} \langle A_\mu x, y \rangle + \overline{\beta} \langle A_\mu x', y \rangle \\ & = \langle \alpha A_\mu x, y \rangle + \langle \beta A_\mu x', y \rangle. \end{split} \] That is, $A_\mu (\alpha x + \beta x') = \alpha A_\mu x + \beta A_\mu x'.$

Remark 2.8. The bra–ket notation so loved by physicists is related to this theorem. We have shown that any sesquilinear pairing on, say, $\mathbb{C}^n$ can be given in terms of a matrix $A \in \operatorname{Mat}_n(\mathbb{C})$ and the usual dot product. A physicist might denote such a pairing by $\langle x \mid A \mid y \rangle.$

Definition 2.26 (Adjoint operators) (Debnath and Mikusinski 2005, 158; Petersen 2012, 242) Let $L$ be a linear operator on a finite-dimensional inner product space $V.$ Then the operator $L^*: V \to V$ satisfying \[ \langle x, Ly \rangle = \langle L^*x, y \rangle, \text{ for all } x, y \in V, \] whose existence and uniqueness is guaranteed by Theorem 2.12, is called the adjoint of $L.$

Proposition 2.10 If $V$ is a finite-dimensional inner product space and $A, B: V \to V,$ then \[ (AB)^* = B^* A^*. \] Moreover, $\mathbb{I}^* = \mathbb{I},$ $(A^*)^* = A,$ and \[(\alpha A + \beta B)^* = \overline{\alpha} A^* + \overline{\beta} B^*\] for all $\alpha, \beta \in F.$

Example 2.32 (Transpose) If we equip $\mathbb{R}^n$ with the standard dot product and $A \in \operatorname{Mat}_n(\mathbb{R}),$ then the adjoint is given by the matrix transpose: $A^* = A^\top.$ This follows because \[ \langle x, Ay \rangle = x^\top Ay = (A^\top x)^\top y = \langle A^\top x, y \rangle. \]

Example 2.33 (Conjugate transpose) If we equip $\mathbb{C}^n$ with the Hermitian dot product and $A \in \operatorname{Mat}_n(\mathbb{C}),$ then the adjoint is given by the conjugate transpose: $A^* \coloneqq \overline{A^\top}.$

Self-Adjoint and Unitary Operators

Definition 2.27 (Self-adjoint operators) (Debnath and Mikusinski 2005, 159; Petersen 2012, 265) A linear operator $A$ on a finite-dimensional inner product space $V$ is called self-adjoint if $L^* = L,$ i.e., if \[ \langle Lx, y \rangle = \langle x, Ly \rangle, \text{ for all } x,y \in V. \] Especially if $F = \mathbb{R},$ we might also call such operators symmetric; if $F = \mathbb{C},$ we call them Hermitian. Beware that, in more general (infinite-dimensional) contexts, all three of these terms have subtly distinct meanings: see Debnath and Mikusinski (2005).

Example 2.34 The operator $(Tf)(t) = t\ f(t)$ is (formally)¹ symmetric with respect to the inner product \[ \langle f, g \rangle \coloneqq \int_0^1 f(t) g(t) \mathop{}\!\mathrm{d}{t}. \]

Example 2.35 A matrix $A \in \operatorname{Mat}_n(\mathbb{R})$ is symmetric (with respect to the dot product) if $A = A^\top.$ Similarly, $B \in \operatorname{Mat}_n(\mathbb{C})$ is Hermitian (with respect to the complex dot product) if $B = B^*$ (cf. Example 2.32 and Example 2.33).

Example 2.36 Take $V$ to be the space of smooth functions $f:[0,1] \to \mathbb{C}$ where $f(0) = f(1)$ together with the inner product \[ \langle f, g \rangle \coloneqq \int_0^1 \overline{f(t)} g(t) \mathop{}\!\mathrm{d}{t} \] The “momentum” operator $A: V \to V$ given by $Af(t) = -i \frac{\mathop{}\!\mathrm{d}{f}}{\mathop{}\!\mathrm{d}{t}}$ is Hermitian. Indeed: \[ \begin{aligned} \langle f, Ag \rangle & = \int_0^1 \overline{f(t)} (-i) g'(t) \mathop{}\!\mathrm{d}{t} \\ & = \underbrace{\left. -i \overline{f(t)}g(t)\right|_0^1}_{=0} + \int_0^1 \overline{(-i) f'(t)} g(t) \mathop{}\!\mathrm{d}{t} \\ & = \langle Af, g \rangle, \end{aligned} \] after integrating by parts.

Definition 2.28 (Unitary operators) (Debnath and Mikusinski 2005, 167; Petersen 2012, 273) An invertible linear operator $L$ on an inner product space $V$ is called unitary if $L^{-1} = L^*.$ If $F = \mathbb{R},$ we instead say orthogonal. In other words, $L$ is unitary if, for all $x,y \in V,$ we have \[ \langle Lx, Ly \rangle = \langle x, y \rangle,\] i.e., that $L$ leaves the inner product invariant. Equivalently, $L$ is unitary if \[ \langle Lx, y \rangle = \langle x, L^{-1}y \rangle. \] for all $x, y \in V$.

Definition 2.29 A matrix $R \in \operatorname{GL}_n(\mathbb{R})$ is orthogonal (with respect to the dot product) if and only if $R^{-1} = R^\top.$ Similarly, $Q \in \operatorname{GL}_n(\mathbb{C})$ is unitary (with respect to the complex dot product) if and only if $Q^{-1} = Q^*.$ This is equivalent to asking that the matrix columns (equivalently, the matrix rows) are orthonormal. (cf. Example 2.32 and Example 2.33).

2.3 Spectral Decomposition

Definition 2.30 (Eigenvectors and eigenvalues) (Bretscher 2013, 310; Petersen 2012, 133) If $F$ is any field and $V$ is an $F$-vector space with $L: V \to V$ a linear map, we say that a nonzero $x \in V$ is an eigenvector of $L$ with eigenvalue $\lambda \in F$ if \[ Lx = \lambda x. \]

Example 2.37 Every non-zero $x \in V$ is an eigenvector of $\mathbb{I}: V \to V$ with eigenvalue $1.$

Example 2.38 The function $f(t) = e^t$ is an eigenvector of $\mathrm{D}= \frac{\mathop{}\!\mathrm{d}{}}{\mathop{}\!\mathrm{d}{t}}$ with eigenvalue $1.$ More generally, the function $f_\lambda(t) = e^{\lambda t}$ is an eigenvector of $\mathrm{D}$ with eigenvalue $\lambda.$

Example 2.39 The matrix $\begin{psmallmatrix} 0 & -1 \\ 1 & 0 \end{psmallmatrix}$ has no real eigenvectors, yet $\begin{psmallmatrix} 0 & -1 \\ 1 & 0 \end{psmallmatrix} \begin{psmallmatrix} 1 \\ i \end{psmallmatrix} = -i \begin{psmallmatrix} 1 \\ i \end{psmallmatrix}.$

Example 2.40 A diagonal matrix $D = \operatorname{Diag}(\lambda_1,\dots,\lambda_n)$ has each standard basis element $e_i$ as an eigenvector with eigenvalue $\lambda_i.$

Example 2.41 The operator $T: C^\infty([0,1]) \to C^\infty([0,1])$ given by $(Tf)(t) \coloneqq t\ f(t)$ (see Example 2.34) has no eigenvalues.

Lemma 2.2 (Characteristic polynomial) (Bretscher 2013, 329; Petersen 2012, 367) If $L$ is a linear operator on a vector space $V,$ then $\lambda \in F$ is an eigenvalue of $L$ if and only if $L-\lambda \mathbb{I}$ has a non-trivial kernel. If $V$ is finite-dimensional, the eigenvalues of $L$ correspond to roots of the polynomial \[ p_L(t) \coloneqq \det(t \mathbb{I}- L) \in F[t], \] called the characteristic polynomial of $L.$ Note that $\deg p_L(t) = \dim V,$ so $L$ can have at most $n$ distinct eigenvalues. Moreover, if $F$ is algebraically closed (namely, $F = \mathbb{C}$), then every linear map has at least one eigenvalue.

Theorem 2.13 If $A \in \operatorname{Mat}_n(F)$ and $S \in \operatorname{GL}_n(F),$ $A$ and $S^{-1} A S$ have the same eigenvalues.

Proof. In fact, conjugate matrices have the same characteristic polynomial: \[ p_{S^{-1} A S }(\lambda) = \det(\lambda \mathbb{I}- S^{-1} A S)= \det(S^{-1} (\lambda \mathbb{I}- A ) S) = \det(\lambda \mathbb{I}- A) = p_A(\lambda). \]

Theorem 2.14 If $L$ is an operator on a finite-dimensional inner product space $V,$ then $\lambda$ is an eigenvalue of $L$ if and only if $\overline{\lambda}$ is an eigenvalue of $L^*.$

Proof (Sketch). Note that $\lambda$ is not an eigenvalue of $(\lambda \mathbb{I}- L)$ if and only if \[ (\lambda \mathbb{I}- L)A = \mathbb{I} \] for some operator $A: V \to V,$ i.e., if $\lambda \mathbb{I}- L$ is invertible. Taking adjoints gives \[ A^* (\overline{\lambda} \mathbb{I}- L^*) = \mathbb{I}, \] and so we see that this condition is equivalent to $\overline{\lambda}$ not being an eigenvalue of $L^*.$

Corollary 2.5 If $L: V \to V$ is a self-adjoint operator on a finite-dimensional inner product space $V,$ then all eigenvalues of $L$ are real.

Proof (Alternative). If $Lx = \lambda x$ with $x$ nonzero, we see directly that \[ \lambda \|x\|^2 = \langle x, \lambda x \rangle = \langle x, Lx \rangle = \langle Lx, x \rangle = \langle \lambda x, x \rangle = \overline{\lambda} \|x\|^2 \] and hence we must have $\lambda \in \mathbb{R}.$

Corollary 2.6 If $L: V \to V$ is a unitary operator on a finite-dimensional inner product space $V,$ then all eigenvalues $\lambda$ of $L$ are complex numbers with $|\lambda| = 1.$

Proof (Alternative). If $Lx = \lambda x$ with $x$ nonzero, we have \[ |\lambda|^2 \|x\|^2 = \langle \lambda x, \lambda x \rangle = \langle Lx, Lx \rangle = \langle x, x \rangle = \|x\|^2 \] and hence we must have $|\lambda|=1.$

Diagonalization

Definition 2.31 If a linear operator $L$ on a finite-dimensional vector space $V$ admits a basis $\mathscr{B} = \{v_1,\dots,v_n\}$ of eigenvectors, a.k.a. an eigenbasis, then $[L]_{\mathscr{B}}$ is a diagonal matrix whose entries are eigenvalues. More precisely, there is a decomposition of $V$ into subspaces, \[ V = \operatorname{Span}\{v_1\} \oplus \cdots \oplus \operatorname{Span}\{v_n\}, \] such that $L = \lambda_1 \oplus \cdots \oplus \lambda_n,$ where $L v_i = \lambda_i v_i.$ We say that $L$ is diagonalizable.

Definition 2.32 If $L: V \to V$ is an operator and $\lambda \in F$ is an eigenvalue, we write \[ E_\lambda = \{ x \in V: Lx = \lambda x \} \] for the subspace all corresponding eigenvectors, called the eigenspace of $\lambda.$

Lemma 2.3 If $L$ is a linear transformation on a vector space $V$ and $\lambda, \lambda' \in F$ are distinct eigenvalues of $L,$ then $E_\lambda \cap E_{\lambda'} = \{0\}.$

Proof (Sketch). If $x \in E_\lambda \cap E_{\lambda'},$ then $\lambda x = Lx = \lambda' x.$ Thus $(\lambda - \lambda') x = 0$ yet $\lambda \not = \lambda',$ so $x = 0.$

Indeed, a linear transformation is diagonalizable if and only if its eigenspaces span $V.$

Corollary 2.7 If $L$ is a linear transformation on an $n$-dimensional vector space $V$ and $L$ has $n$ distinct eigenvalues, then $L$ is diagonalizable.

Proof (Sketch). If $L$ has $n$ distinct eigenvalues $\lambda_1, \dots, \lambda_n,$ then we have $n$ corresponding eigenspaces $E_{\lambda_i}$. Each of these must be at least $1$-dimensional, and their direct sum is at most $n$-dimensional by Corollary 2.1 since \[ E_{\lambda_1} \oplus \cdots \oplus E_{\lambda_n} \leq V. \] It follows that $\dim E_{\lambda_i} = 1$ for each $i$, and the above inclusion is actually equality.

Example 2.42 The matrix $\begin{psmallmatrix} \lambda & 1 \\ 0 & \lambda \end{psmallmatrix}$ is not diagonalizable for any choice of $\lambda \in F,$ since it only has a single eigenvalue, namely $\lambda,$ and yet $E_\lambda = \operatorname{Span}\{ e_1 \} < F^2.$

Remark 2.9. In a sense, most operators are diagonalizable. Briefly, given a matrix \[ A = \begin{pmatrix} \alpha_{1,1} & \cdots & \alpha_{1,n} \\ \vdots & \ddots & \vdots \\ \alpha_{n,1} & \cdots & \alpha_{n,n} \\ \end{pmatrix} \in \operatorname{Mat}_n(F), \] the characteristic polynomial has $n$ roots when counted with multiplicity. Based on the theory of resultants, there is a polynomial in the $n^2$ entries of $A$ which is zero if and only if $A$ has a repeated eigenvalue (in the proper splitting field). The zero sets of non-trivial polynomials are very small compared to all of $\operatorname{Mat}_n(F) = F^{n^2}$; over $F = \mathbb{R}$ or $F = \mathbb{C},$ the set of non-diagonalizable matrices have Lebesgue measure zero. For more, see (Dummit and Foote 2003, 619).

Example 2.43 The matrix $\begin{psmallmatrix} \alpha & \beta \\ \gamma & \delta \end{psmallmatrix}$ has distinct eigenvalues so long as $(\alpha-\delta)^2+4\beta \gamma \not = 0.$

Theorem 2.15 Suppose $A$ and $B$ are diagonalizable linear operators on a finite-dimensional vector space $V.$ If $AB = BA$ then the two operators are simultaneously diagonalizable, i.e., there is a choice of basis $\mathscr{B} \subset V$ so that $[A]_{\mathscr{B}}$ and $[B]_{\mathscr{B}}$ are both diagonal matrices.

Proof. Since $A$ is diagonalizable, we have a decomposition $V = E_{\lambda_1} \oplus \cdots \oplus E_{\lambda_k}.$ Next, given $x \in E_{\lambda_i},$ we have $ABx = BAx = B (\lambda_i x) = \lambda_i Bx,$ i.e., $Bx$ is still an eigenvector of $A$ with eigenvalue $\lambda_i.$ This means that $B$ restricts to an operator $B_i$ on each eigenspace $E_{\lambda_i}$ of $A.$ Hence, in any eigenbasis $\mathscr{A}$ for $A,$ we have $A$ diagonal and \[ [B]_{\mathscr{A}} = \begin{pmatrix} B_1 & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & B_k \end{pmatrix}, \] a block diagonal matrix. Since $B$ is also diagonalizable, we can choose eigenbases $\mathscr{B}_i \subset E_{\lambda_i}$ to diagonalize each $B_i$; therefore $\mathscr{B} = \mathscr{B}_1 \cup \cdots \cup \mathscr{B}_k$ is an eigenbasis for $B.$ Remembering that each element of $\mathscr{B}$ is an eigenvector of $A,$ we see that $[A]_{\mathscr{B}}$ and $[B]_{\mathscr{B}}$ are both diagonal.

Example 2.44 The matrices $A = \begin{psmallmatrix} -7 & 12 & 12 \\ 2 & -5 & -4 \\ -6 & 12 & 11 \end{psmallmatrix}$ and $B = \begin{psmallmatrix} 0 & 6 & 2 \\ -2 & 6 & 4 \\ 2 & -3 & -3 \end{psmallmatrix}$ commute. Via $S = \begin{psmallmatrix} 2 & 2 & 3 \\ 0 & 1 & -1 \\ 1 & 0 & 3 \end{psmallmatrix},$ we have \[ S^{-1} A S = \begin{pmatrix} -1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \] and \[ S^{-1} B S = \begin{pmatrix} 1 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 0 \end{pmatrix}, \] i.e., changing bases to $\left( \begin{psmallmatrix} 2 \\ 0 \\ 1 \end{psmallmatrix}, \begin{psmallmatrix} 2 \\ 1 \\ 0 \end{psmallmatrix}, \begin{psmallmatrix} 3 \\ -1 \\ 3 \end{psmallmatrix} \right)$ is sufficient to diagonalize $A$ but not $B,$ yet $B$ has distinct eigenvalues and so must be diagonalizable! Instead, take $R = \begin{psmallmatrix} 2 & 4 & 3 \\ 0 & 1 & -1 \\ 1 & 1 & 3 \end{psmallmatrix}$: \[ R^{-1} A R = \begin{pmatrix} -1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \] and \[ R^{-1} B R = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 0 \end{pmatrix}. \]

Theorem 2.16 (Bretscher 2013, 334; Petersen 2012, 368) If $L$ is a linear map on some finite-dimensional inner product space $V$ with $\lambda_1, \dots, \lambda_n \in F$ the roots of $p_L(\lambda)$ counted with multiplicity, then \[ \det(L) = \lambda_1 \cdots \lambda_n \quad \text{ and } \quad \operatorname{Tr}(L) = \lambda_1 + \cdots + \lambda_n. \] Said more simply: the determinant is the product of eigenvalues and the trace is their sum.

Proof (Favorite sketch). If $L: V \to V$ is diagonalizable, then we can find a basis $\mathscr{B}$ such that $[L]_{\mathscr{B}}$ is diagonal; the determinant (or trace) of a diagonal matrix is just the product (or sum, respectively) of the diagonal entries. So the result is clear for diagonalizable operators.

Moreover, because “most” operators are diagonalizable in the previously alluded sense (Remark 2.9), even a non-diagonalizable $L: V \to V$ is arbitrarily “close” to a diagonalizable operator—more precisely, the set of diagonalizable operators is dense amongst all operators. As the operations of determinant and trace are continuous, the general formulas follow.

Theorem 2.17 If $P: V \to V$ is a projection on a finite-dimensional vector space $V,$ then its only possible eigenvalues are $0$ and $1.$ Moreover, $\operatorname{Tr}(P)$ is an integer and is equal to the dimension of the subspace projected onto by $P.$

We will prove this result in Exercise B.1.

Definition 2.33 A linear transformation $L$ on a finite-dimensional inner product space $V$ is normal if it commutes with its adjoint, i.e., $[L,L^*] = 0.$

Example 2.45 All self-adjoint and unitary operators are normal.

Remark 2.10. Self-adjoint maps are analogous to real numbers in the algebra of operators. In fact, any operator $L$ can be written uniquely as the sum of “real” and “imaginary” part, $L = A+Bi,$ where $A$ and $B$ are both self-adjoint. This is accomplished by taking \[ A = \tfrac{1}{2}(L+L^*) \] and \[ B = \tfrac{1}{2i}(L-L^*). \] Moreover, one can check that $[L^*,L] = 2i[A,B],$ i.e., $L$ is normal if and only if its real and imaginary parts commute. Thus, if we think of self-adjoint operators as behaving like real numbers, with normal transformations roughly analogous to complex numbers. We will later see that unitary operators behave like complex numbers of modulus 1.

Spectral Theory

Lemma 2.4 Let $L$ be a normal operator on an inner product space $V.$ Then $x \in V$ is an eigenvector of $L$ with eigenvalue $\lambda \in F,$ i.e., $Lx = \lambda x,$ if and only if $L^* x = \overline{\lambda} x.$

Proof. First we show that $L$ being normal means that $L$ and $L^*$ have the same nullspace: \[ \|Lx\|^2 = \langle Lx, Lx \rangle = \langle L^* L x, x \rangle = \langle LL^* x, x \rangle = \langle L^* x, L^* x \rangle = \|L^*x\|^2, \] so the left hand side is zero if and only if the right hand side is. Moreover, $L - \lambda \mathbb{I}$ is normal: \[ \begin{split} (L - \lambda \mathbb{I})(L - \lambda \mathbb{I})^* & = (L - \lambda \mathbb{I})(L^* - \overline{\lambda} \mathbb{I}) \\ & = LL^* - \lambda L^* - \overline{\lambda} L + |\lambda|^2 \mathbb{I}\\ & = L^*L - \overline{\lambda} L - \lambda L^* + |\lambda|^2 \mathbb{I}\\ & = (L^* - \overline{\lambda} \mathbb{I})(L - \lambda \mathbb{I}) = (L - \lambda \mathbb{I})^*(L - \lambda \mathbb{I}). \end{split} \] Thus $(L-\lambda \mathbb{I})x = 0$ if and only if $(L^*-\overline{\lambda}\mathbb{I})x = 0,$ i.e., $L x = \lambda x$ if and only if $L^* x = \overline{\lambda} x.$

Theorem 2.18 If $L$ is a normal operator on an inner product space $V$ with $\lambda, \lambda' \in F$ distinct eigenvalues of $L,$ then the eigenspaces $E_\lambda$ and $E_{\lambda'}$ are orthogonal, i.e., we have $\langle x, y \rangle = 0$ for all $x \in E_\lambda$ and $y \in E_{\lambda'}.$

Proof. For any $x \in E_\lambda$ and $y \in E_{\lambda'},$ we have \[ \lambda' \langle x, y \rangle = \langle x, \lambda' y \rangle = \langle x, Ly \rangle = \langle L^* x, y \rangle = \langle \overline{\lambda} x, y \rangle = \lambda \langle x, y \rangle. \] Since $(\lambda-\lambda') \langle x, y \rangle = 0$ and $\lambda \not = \lambda',$ we must always have $\langle x, y \rangle = 0.$

Theorem 2.19 (Spectral theorem) (Debnath and Mikusinski 2005, 196; Petersen 2012, 275) If $L$ is a linear map on a finite-dimensional inner product space $V,$ then $L$ is normal if and only if there is an orthonormal basis $\{v_1,\dots,v_n\} \subset V$ consisting of eigenvectors of $L,$ i.e., $L v_i = \lambda_i v_i.$ Thus, for any $x \in V,$ \[ Lx = \sum_{i=1}^n \lambda_i \langle v_i, x \rangle v_i. \]

Corollary 2.8 If $A \in \operatorname{Mat}_n(F)$ is self-adjoint (symmetric if $F = \mathbb{R}$) or unitary (orthogonal if $F = \mathbb{R}$), then there is a unitary matrix $Q$ whose columns are eigenvectors of $A$ with \[ Q^* A Q = \operatorname{Diag}(\lambda_1,\dots,\lambda_n) \] Indeed, if a matrix $A$ satisfies the above, then it is necessarily normal. Since $(Q^* A Q)^* = Q^* A^* Q$, we have \[ \begin{split} A A^* & = (Q \operatorname{Diag}(\lambda_1,\dots,\lambda_n) Q^*)(Q \operatorname{Diag}(\overline{\lambda_1},\dots,\overline{\lambda_n}) Q^*) \\ & = (Q \operatorname{Diag}(|\lambda_1|^2,\dots,|\lambda_n|^2) Q^*) \\ & = (Q \operatorname{Diag}(\overline{\lambda_1},\dots,\overline{\lambda_n}) Q^*) (Q \operatorname{Diag}(\lambda_1,\dots,\lambda_n) Q^*) \\ & = A^* A. \end{split} \]

In this sense, the Spectral Theorem provides an alternate definition of normal matrices—they are exactly those which admit a unitary diagonalization.

Example 2.46 Consider the “momentum” operator $A = -i \frac{\mathop{}\!\mathrm{d}{}}{\mathop{}\!\mathrm{d}{t}}$ (Example 2.36) on the vector space \[ V = \operatorname{Span}\{ 1, \cos t, \sin t \} \] with the inner product from Equation 2.3. In the given basis, \[ A = \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & -i \\ 0 & i & 0 \end{pmatrix} \] is evidently self-adjoint. We compute $p_A(\lambda) = \lambda^3-\lambda,$ which has the roots $\lambda = 0, 1, -1$; these are the eigenvalues of $A.$ To find the eigenvectors, we compute the nullspace of \[ A = \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & -i \\ 0 & i & 0 \end{pmatrix}, A-\mathbb{I}= \begin{pmatrix} -1 & 0 & 0 \\ 0 & -1 & -i \\ 0 & i & -1 \end{pmatrix}, \text{ and } \quad A+\mathbb{I}= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & -i \\ 0 & i & 1 \end{pmatrix}. \] We can do this via the application of echelon form (Bretscher 2013, 116) or by inspection: \[ \ker A = \operatorname{Span}\left\{ \begin{psmallmatrix} 1 \\ 0 \\ 0 \end{psmallmatrix} \right\}, \ker (A-\mathbb{I}) = \operatorname{Span}\left\{ \begin{psmallmatrix} 0 \\ 1 \\ i \end{psmallmatrix} \right\}, \text{ and } \ker (A+\mathbb{I}) = \operatorname{Span}\left\{ \begin{psmallmatrix} 0 \\ 1 \\ -i \end{psmallmatrix} \right\}. \] So, the eigenvectors of $A$ are $1,$ $\cos t + i \sin t = e^{it},$ and $\cos t - i \sin t = e^{-it},$ which can be normalized to the eigenbasis \[ \mathscr{B} = \left( \tfrac{1}{\sqrt{2}}, \tfrac{1}{\sqrt{2}} e^{it}, \tfrac{1}{\sqrt{2}}e^{-it} \right). \] If we wish to compute the result of $A$ applied to $x=3 + 2 \cos t,$ we can first use inner products to write $x$ in this eigenbasis: \[ x = \underbrace{\left( \tfrac{1}{\pi} \int_0^{2\pi} x(s) \tfrac{1}{\sqrt{2}} \mathop{}\!\mathrm{d}{s} \right)}_{3 \sqrt{2}} \tfrac{1}{\sqrt{2}} + \underbrace{\left( \tfrac{1}{\pi} \int_0^{2\pi} x(s) \tfrac{e^{-is}}{\sqrt{2}} \mathop{}\!\mathrm{d}{s} \right)}_{\sqrt{2}} \tfrac{e^{it}}{\sqrt{2}} + \underbrace{\left( \tfrac{1}{\pi} \int_0^{2\pi} x(s) \tfrac{e^{is}}{\sqrt{2}} \mathop{}\!\mathrm{d}{s} \right)}_{\sqrt{2}} \tfrac{e^{-it}}{\sqrt{2}} \] Hence we compute \[ \begin{aligned} Ax & = 3 \sqrt{2} A(\tfrac{1}{\sqrt{2}}) + \sqrt{2} A(\tfrac{1}{\sqrt{2}} e^{it}) + \sqrt{2} A(\tfrac{1}{\sqrt{2}}e^{-it}) \\ & = 3 \sqrt{2} \cdot 0 \cdot (\tfrac{1}{\sqrt{2}}) + \sqrt{2} \cdot 1 \cdot (\tfrac{1}{\sqrt{2}} e^{it}) + \sqrt{2} \cdot (-1) \cdot (\tfrac{1}{\sqrt{2}}e^{-it}) \\ & = e^{it} - e^{-it} = 2i \sin t. \end{aligned} \] Applying $A$ is easy when $x$ is written in the eigenbasis—simply scaling the individual eigenvectors by their eigenvalues—because it is diagonalized by this basis: $[A]_{\mathscr{B}} = \operatorname{Diag}(0,1,-1).$ Compare this with Remark 2.7.

We are being reckless: the associated vector spaces are infinite-dimensional, so defining adjoints takes greater care. See Debnath and Mikusinski (2005) for a more cautious approach.↩︎