Extreme Value Theorem
If \(I\) is a closed bounded interval and \(f:I \rightarrow \mathbb{R}\) is continuous on \(I\), then \(f\) is bounded on \(I\). Moreover, if $$ \begin{align*} M = \sup_{x \in I} f(x) \quad \text{ and } \quad m = \inf_{x \in I} f(x), \end{align*} $$ then there exists points \(x_M, x_m \in I\) such that $$ \begin{align*} f(x_M) = M \quad \text{ and } \quad f(x_m) = m. \end{align*} $$

Strategy

The first part of this proof is showing that \(f\) must be bounded. The trick to this is using Bolzano-Weierstrass’s Theorem together with Continuity. In a nutshell, since \(f\) is unbounded, then we can build a sequence \(\{x_n\}\) such that for all \(n \in \mathbb{N}\), we can find an \(x_n \in I\) that satisfies \(|f(x_n)| > n\). At the same time, since this interval is bounded where \(a \leq x_{n} \leq b\), then by Bolzano-Weierstrass, we must have a convergent subsequence \(x_{n_k}\) that converges to some limit \(x_0\). By Continuity (sequential criterion), we must also have that \(|f(x_{n_k})| \to |f(x_0)|\). But a convergent sequence is also bounded. Hence, \(f(x_{n_k})\) is bounded. This is a contradiction since \(f(x_{n_k}) > n_k > k\) for all \(k\) which implies that \(f(x_{n_k}) \to \infty\).

The second part of the proof replies on the Completeness Axiom. Since \(f\) is bounded on \(I\), then the set of all points \(f(x)\) where \(x \in [a,b]\) is bounded. Hence by completeness, \(M = \sup_I f(x)\) exists. By the definition of the supremum and for any \(n \in \mathbb{N}\), we have \(M - \frac{1}{n} < f(x_n) \leq M\). This sequence \(\{f(x_n)\}\) converges to \(M\) by the Squeeze Theorem. At the same time, since \(\{x_n\} \in I\) and \(I\) is bounded, then by Bolzano-Weierstrass, we have a convergent subsequence \(\{x_{n_k}\}\) that converges to some point, call it \(x_M\). And again by Continuity (sequential criterion), we must also have that \(f(x_{n_k}) \to f(x_M)\). But now since the main sequence \(\{f(x_n)\}\) converges to \(M\), then we must have that every subsequence converges to \(M\) as well. Hence, \(f(x_M) = M\).

Proof

Suppose for the sake of contradiction that \(f\) is unbounded on \(I\). Then by definition of unbounded, for every \(M > 0 \in \mathbb{R}\), we can find some \(x \in I\) such that \(|f(x)| > M\). In particular (if we discretize this), then for every natural number \(n \in \mathbb{N}\), we can find a point \(x_n \in I\) such that

$$ \begin{align*} |f(x_n)| > n \quad \quad \quad (1) \end{align*} $$

Thus, we obtain a sequence \(\{x_n\}\) such that \(|f(x_n)| \to \infty\) as \(n \to \infty\). But recall that \(\{x_n\} \in I\) and we’re given that \(I\) is bounded. Hence, by the Bolzano-Weierstrass Theorem, there exists a convergent subsequence \(x_{n_k}\) such that

$$ \begin{align*} x_{n_k} \to x_0 \quad \text{ as } \quad k \to \infty \end{align*} $$

But note that \(I\) is a closed interval so \(a \leq x_{n_k} \leq b\) for all \(x_{n_k}\). Hence by the Comparison Theorem, we must have that

$$ \begin{align*} a \leq x_0 \leq b \end{align*} $$

But we are also given that \(f\) is continuous on \(I\) so \(f\) is continuous at \(x_0\). Thus, since \(x_{n_k} \to x_0\) and \(x_{n_k} \in I\), then by the Sequential Characterization of Continuity Theorem

$$ \begin{align*} f(x_{n_k}) \to f(x_0) \quad \text{ as } \quad k \to \infty \end{align*} $$

which also implies that

$$ \begin{align*} |f(x_{n_k})| \to |f(x_0)| \quad \text{ as } \quad k \to \infty \end{align*} $$

But this means by definition that when taking \(\epsilon = 1\), that there exists a \(K_0 \in \mathbb{N}\) such that for all \(k \geq K_0\)

$$ \begin{align} |f(x_{n_k})| - |f(x_0)| &\leq |f(x_{n_k}) - f(x_0)| < 1 \end{align} $$

This implies

$$ \begin{align} |f(x_{n_k})| \leq |f(x_0)| + 1 \quad \text{ for all } k \geq K_0 \quad \quad \quad (2) \end{align} $$

So for all sufficiently large \(k\), \(|f(x_{n_k})|\) is bounded above by \(|f(x_0)| + 1\). However, by (1), we have

$$ \begin{align*} |f(x_{n_k})| > n_k \quad \text{ for all } k \end{align*} $$

This means that as \(n_k \to \infty\), then

$$ \begin{align*} |f(x_{n_k})| \to \infty \end{align*} $$

This is a contradiction since by (2), we know that \(|f(x_{n_k})|\) is bounded above for all \(k > K_0\). Thus, \(f\) must be bounded on \(I\).

Now since \(f\) is bounded on \(I\), then by the completeness axiom, \(\sup_{x \in I} f(x)\) exists. Let \(M = \sup_{x \in I} f(x)\). By definition of the supremum, this means that for all \(n \in \mathbb{N}\), there exists an \(x_n \in I\) such that

$$ \begin{align*} M - \frac{1}{n} < f(x_n) \leq M \quad \quad \quad (3) \end{align*} $$

Since \(I\) is closed and bounded, the sequence \(\{x_n\} \subset I\) is also bounded. Hence, by the Bolzano-Weierstrass Theorem, there exists a convergent subsequence \(x_{n_k} \in I\) and a point \(x_M \in I\) such that

$$ \begin{align*} x_{n_k} \to x_{M} \quad \text{ as } \quad k \to \infty \end{align*} $$

But \(f\) is continuous on \(I\). Then by the Sequential Characterization of Continuity Theorem

$$ \begin{align*} f(x_{n_k}) \to f(x_M) \end{align*} $$

If we take the limit of (3), then note that as \(k \to \infty\), \(n_k \to \infty\) and thus \(\frac{1}{n_k} \to 0\). Then the left hand side approaches \(M\). The middle term approaches \(f(x_M)\) and the right hand side approaches \(M\) since it’s a constant. Hence

$$ \begin{align*} M \leq f(x_M) \leq M \end{align*} $$

Thus, by the Squeeze Theorem,

$$ \begin{align*} f(x_M) = M = \sup_{x \in I} f(x). \ \blacksquare \end{align*} $$

EVT Conditions

EVT Condition 1
[Lecture 10] How does EVT fail when \(f\) is not continuous?

Let’s take \(f(x) = \frac{1}{x}\) on \([0,1]\). Note here that we can’t really talk about continuity at \(x = 0\) since \(f\) is not even defined on \(x = 0\). Thus, define \(f\) as follows

$$ \begin{align} f(x) = \begin{cases} \frac{1}{x}, & \text{if } 0 < x \leq 1, \\ 0, & \text{if } x = 0. \end{cases} \end{align} $$

We showed here that \(f\) is not continuous at \(x = 0\). Now, notice that \(f\) doesn’t attain a maximum in \([0,1]\). It is unbounded since as \(x \to 0\), \(f(x) \to \infty\). Hence, the continuity condition is essential.

EVT Condition 2
[Lecture 10] How does EVT fail when \([a,b]\) is not bounded?

[TODO]

EVT Condition 3
[Lecture 10] How does EVT fail when \([a,b]\) is not closed?

Then, we might not attain the maximum inside the interval. Take \(f(x)=x^2\) and the interval \((0,2)\). The maximum is \(f(2) = 4\) but it’s not attained!

References