Fundamental Theorem of Hilbert Space
Suppose that \(\{\mathbf{e}_n\}\) is an orthonormal set for the Hilbert
space \(H\). Then the following are equivalent.
- (Completeness) \(\langle \mathbf{u},\mathbf{e}_n\rangle=0\)
for all \(n \Longrightarrow \mathbf{u}=0\).
- (Totality) \(\displaystyle
\mathbf{u}=\sum_{n=1}^{\infty}
\langle\mathbf{u},\mathbf{e}_n\rangle\mathbf{e}_n\)
for all \(\mathbf{u}\in H\).
Equivalently, if we let \(S_n = p_N(u) = \sum_{n=1}^{N}
\langle\mathbf{u},\mathbf{e}_n\rangle\mathbf{e}_n \), then as \(N \to \infty\)
\[ \|\mathbf{u} - S_N\| \to 0 \]
for all \(u \in H\)
- (Parseval identity) \(\displaystyle
\sum_{n=1}^{\infty}\bigl|\langle\mathbf{u},\mathbf{e}_n\rangle\bigr|^2
=\|\mathbf{u}\|^2\)
Proof. \((1) \to (2)\). Take \(u\in H\) and let
$$
\begin{align*}
S_N = p_N(u) = \sum_{n=1}^{N}\langle u,e_n\rangle e_n.
\end{align*}
$$
We want to show that \(S_N\to u\). First, we show that \((S_N)\) is Cauchy. Let \(N>M\). Then \(S_N-S_M = \sum_{n=M+1}^{N} \langle u,e_n\rangle e_n\). Taking the squared norm of both sides, we get
$$
\begin{align*}
\|S_N-S_M\|^2
&=\left\|\sum_{n=M+1}^{N}\langle u,e_n\rangle e_n\right\|^2\\
&=\left\langle
\sum_{n=M+1}^{N}\langle u,e_n\rangle e_n,
\sum_{k=M+1}^{N}\langle u,e_k\rangle e_k
\right\rangle\\
&=\sum_{n=M+1}^{N}\langle u,e_n\rangle
\left\langle e_n,\sum_{k=M+1}^{N}\langle u,e_k\rangle e_k\right\rangle\\
&=\sum_{n=M+1}^{N}\sum_{k=M+1}^{N}
\langle u,e_n\rangle\overline{\langle u,e_k\rangle}
\langle e_n,e_k\rangle.
\end{align*}
$$
By orthonormality, \(\langle e_n,e_k\rangle=0\) when \(n\ne k\), and \(\langle e_n,e_n\rangle=1\). Therefore,
$$
\begin{align*}
\|S_N-S_M\|^2
&=\sum_{n=M+1}^{N}\langle u,e_n\rangle
\overline{\langle u,e_n\rangle}\\
&=\sum_{n=M+1}^{N}|\langle u,e_n\rangle|^2.
\end{align*}
$$
By Bessel’s inequality, we know that
$$
\begin{align*}
\sum_{n=1}^{\infty}|\langle u,e_n\rangle|^2
\le \|u\|^2<\infty.
\end{align*}
$$
Thus, given \(\varepsilon>0\), for all sufficiently large \(M\) and all \(N>M\),
$$
\begin{align*}
\|S_N-S_M\|^2
=\sum_{n=M+1}^{N}|\langle u,e_n\rangle|^2
<\varepsilon^2.
\end{align*}
$$
Taking square roots gives
$$
\begin{align*}
\|S_N-S_M\|<\varepsilon.
\end{align*}
$$
Therefore, \((S_N)\) is Cauchy. Since \(H\) is complete, then there exists some \(v\in H\) where
$$
\begin{align*}
\boxed{S_N\to v} \tag{A}
\end{align*}
$$
It remains to show that \(v=u\). Recall that (1) states if \(\langle w,e_n\rangle=0\) for every \(n\), then \(w=0\). We want to apply this to \(w=u-v\). Thus, our next goal is to show that
$$
\begin{align*}
\langle u-v,e_n\rangle=0
\qquad\text{for every }n.
\end{align*}
$$
By linearity,
$$
\begin{align*}
\langle u-v,e_n\rangle
=\langle u,e_n\rangle-\langle v,e_n\rangle.
\end{align*}
$$
So we want to show that \(\langle u,e_n\rangle=\langle v,e_n\rangle\). Fix \(n\) and take \(N\ge n\). Then
$$
\begin{align*}
\langle S_N,e_n\rangle
&=\left\langle\sum_{k=1}^{N}\langle u,e_k\rangle e_k,e_n\right\rangle\\
&=\sum_{k=1}^{N}\langle u,e_k\rangle\langle e_k,e_n\rangle.
\end{align*}
$$
Since \(\langle e_k,e_n\rangle=0\) if \(k\ne n\) and \(\langle e_n,e_n\rangle=1\), we obtain
$$
\begin{align*}
\boxed{\langle S_N,e_n\rangle=\langle u,e_n\rangle.} \tag{B}
\end{align*}
$$
Next, since \(S_N\to v\), the Cauchy–Schwarz inequality gives
$$
\begin{align*}
0 &\le |\langle S_N,e_n\rangle-\langle v,e_n\rangle| \quad \text{(by definition it's nonnegative)} \\
&= |\langle S_N-v,e_n\rangle| \quad \text{(by linearity)}\\
&\le \|S_N-v\|\,\|e_n\| \quad \text{(By Cauchy Schwarz)}\\
&=\|S_N-v\|.
\end{align*}
$$
Taking the limit we get
$$
\begin{align*}
0 &\le \lim_{N\to\infty} |\langle S_N,e_n\rangle-\langle v,e_n\rangle| \le \lim_{N\to\infty} \|S_N-v\|.
\end{align*}
$$
The right-hand side tends to zero by (A). By the squeeze theorem,
$$
\begin{align*}
\lim_{N\to\infty}
|\langle S_N,e_n\rangle-\langle v,e_n\rangle|=0.
\end{align*}
$$
When \(N\ge n\), by (B) this becomes
$$
\begin{align*}
\lim_{N\to\infty}
|\langle u,e_n\rangle-\langle v,e_n\rangle|=0.
\end{align*}
$$
The expression inside the limit is constant with respect to \(N\), so
$$
\begin{align*}
|\langle u,e_n\rangle-\langle v,e_n\rangle|&=0,\\
\langle u,e_n\rangle-\langle v,e_n\rangle&=0,\\
\langle u-v,e_n\rangle&=0.
\end{align*}
$$
This holds for every \(n\). By assumption (1), \(u-v=0\), and hence \(u=v\).
Therefore,
$$
\begin{align*}
\boxed{u=\sum_{n=1}^{\infty}\langle u,e_n\rangle e_n.}
\end{align*}
$$
\((2) \to (3)\). Take \(u\in H\) and let
$$
\begin{align*}
S_N=p_N(u)=\sum_{n=1}^{N}\langle u,e_n\rangle e_n.
\end{align*}
$$
By totality,
$$
\begin{align*}
\|u-S_N\|\to0
\qquad\text{as }N\to\infty.
\end{align*}
$$
Now write
$$
\begin{align*}
u=(u-S_N)+S_N.
\end{align*}
$$
Recall that by Theorem 0.1, \(S_N=p_N(u)\) is the orthogonal projection of \(u\) onto \(W_N = \operatorname{span}\{e_1,\ldots,e_N\}\). By definition of the orthogonal projection onto a subspace \(W_N\), we know that \(p_N(u)\in W_N\) and
$$
\begin{align*}
\langle u-p_N(u),w\rangle=0
\qquad\text{for every }w\in W_N.
\end{align*}
$$
Taking \(w=p_N(u)=S_N\) gives
$$
\begin{align*}
\langle u-p_N(u),p_N(u)\rangle&=0,\\
\langle u-S_N,S_N\rangle&=0.
\end{align*}
$$
Therefore, by the Pythagorean theorem,
$$
\begin{align*}
\|u\|^2=\|u-S_N\|^2+\|S_N\|^2.
\end{align*}
$$
and as we have shown in Lemma 0.3, orthonormality gives
$$
\begin{align*}
\|u\|^2 = \|u-S_N\|^2+\sum_{n=1}^{N}|\langle u,e_n\rangle|^2.
\end{align*}
$$
Since \(\|u-S_N\|\to0\) by totality, taking the limit as \(N\to\infty\) gives
$$
\begin{align*}
\lim_{N\to\infty}\sum_{n=1}^{N}|\langle u,e_n\rangle|^2 &= \lim_{N\to\infty}\left(\|u\|^2-\|u-S_N\|^2\right)\\
\sum_{n=1}^{\infty}|\langle u,e_n\rangle|^2 &= \|u\|^2.
\end{align*}
$$
Therefore, Parseval’s identity holds:
$$
\begin{align*}
\boxed{\sum_{n=1}^{\infty}|\langle u,e_n\rangle|^2=\|u\|^2.}
\end{align*}
$$
\((3) \to (1)\). Assume Parseval’s identity holds:
$$
\begin{align*}
\sum_{n=1}^{\infty}|\langle u,e_n\rangle|^2=\|u\|^2
\qquad\text{for every }u\in H.
\end{align*}
$$
Suppose \(u\in H\) satisfies \(\langle u,e_n\rangle=0\) for every \(n\). Then
$$
\begin{align*}
\|u\|^2
=\sum_{n=1}^{\infty}|\langle u,e_n\rangle|^2
=0.
\end{align*}
$$
Since a vector has norm zero only if it is the zero vector, \(u=0\). Thus, (1) holds.
Remarks
Suppose now that we want to apply Parversal’s identity. We are given the inner product space \(L^2([-\pi,\pi])\). We know it is equipped with an orthonormal set \(\{e^{inx}\}_{n\in\mathbb{Z}}\). We also know that it’s complete. So it is a Hilbert space. But now, we still need to show either (1) completeness or (2) totality in order to apply Parversal. It isn’t enough to just have a Hilbert space and an orthonormal set.
References
- SFSU Fourier Analysis by Professor Chun-Kit Lai
- Fourier Analysis: An Introduction by Elias M.Stein and Rami Shakarchi