This page is a sub-page of our page on Linear Algebra.
///////
The sub-pages of this page are:
• …
///////
Related KMR-pages:
• Fourier Series (of periodic functions)
• Fourier Transforms (of non-periodic functions)
///////
Other relevant sources of information:
• Euclidean spaces
• Inner product spaces
• Vector spaces
• Normed vector spaces and inner product spaces
• Topological vector spaces
• Banach spaces
• Hilbert spaces 1.
• Hilbert spaces 2.
• The Stone-Weierstrass theorem.
• Orthonormal basis
• Dual basis
• Fourier series
• Fourier transform
• The Fourier inversion theorem
• Signal reconstruction
• Relation to Fourier series
• Poisson summation formula
• Discrete Fourier transform
Applications of the Fourier transform
• Discrete time Fourier transform
• Fast Fourier transform
• Harmonic analysis
• Parseval’s theorem
• Parseval’s identity
• Plancherel’s theorem
• Lebesgue measure
• The unitarity of the Fourier transform.
• Unitary operator
///////
Quoting Wikipedia (on Sequence spaces):
In functional analysis and related areas of mathematics, a sequence space is a vector space whose elements are infinite sequences of real or complex numbers. Equivalently, it is a function space whose elements are functions from the natural numbers to the field K of real or complex numbers. The set of all such functions is naturally identified with the set of all possible infinite sequences with elements in K, and can be turned into a vector space under the operations of pointwise addition of functions and pointwise scalar multiplication. All sequence spaces are linear subspaces of this space. Sequence spaces are typically equipped with a norm, or at least the structure of a topological vector space.
The most important sequence spaces in analysis are the \, ℓ^p spaces, consisting of the p-power summable sequences, with the p-norm. These are special cases of \, L^p \, spaces for the counting measure on the set of natural numbers. Other important classes of sequences like convergent sequences or null sequences form sequence spaces, respectively denoted \, c \, and \, c_0 , with the sup norm. Any sequence space can also be equipped with the topology of pointwise convergence, under which it becomes a special kind of Fréchet space called FK-space.
/////// End of quote from Wikipedia
\, \mathscr{l} \,
\, \mathscr{L} \,
\, \mathcal{l} \,
\, ℓ \,
\, \ell \,
\, \mathcal{L} \,
Shift of basis for vectors:
Assume that \, \mathbb{C}^m \, is an inner product space over the complex numbers \, \mathbb{C} \,
with the inner product given by \, x \cdot y = x_1 \overline{y_1} + \cdots + x_m \overline{y_m} \, for \, x, y \in \mathbb{C}^m .
Then we have the respective representations
\, [x]_B = { \left< \hat{x} \right> }_B = { \left< \begin{matrix} \hat{x}_1 \\ \vdots \\ \hat{x}_m \end{matrix} \right> }_B \, and \, [x]_{B'} = { \left< \hat{x'} \right> }_{B'} ={ \left< \begin{matrix} \hat{x'}_1 \\ \vdots \\ \hat{x'}_m \end{matrix} \right> }_{B'} ,
and the respective reconstructions
\, x = \left( [x]_B \right)_B = \hat{x}_1 \, b_1 + \cdots + \hat{x}_m \, b_m = \left( [x]_{B'} \right)_{B'} = \hat{x'}_1 \, b'_1 + \cdots + \hat{x'}_m \, b'_m .
In matrix notation, we can write
\, [x]_{B'} = [ \;\; ]_{B'}^B \, [x]_B .
\, [x]_B = [ \;\; ]_B^{B'} \, [x]_{B'} = [ \;\; ]_B^{B'} \, [ \;\; ]_{B'}^B \, [x]_B .
Hence we have
\, [ \;\; ]_B^{B'} \, [ \;\; ]_{B'}^B = I ,
where \, I \, is the identity matrix, and therefore
\, [ \;\; ]_B^{B'} = { [ \;\; ]_{B'}^B }^{-1} .
Now, if \, [ \;\; ]_{B'}^B \, is a unitary matrix, we have
\, { [ \;\; ]_{B'}^B }^{-1} = { [ \;\; ]_{B'}^B }^{*} ,
where the * denotes “complex transposition” of the matrix elements,
which means transposition of them accompanied by their complex conjugation.
///////
\, [f]_x \,
\, M = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{x_k} \\ \scriptsize {\xi}_j \downarrow \, \begin{bmatrix} \, \large e^{2 \pi x \cdot \xi} \, \end{bmatrix} \\ & \end{matrix} \,
\, [\hat{f}]_{{\xi}_j} = \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{x} \\ \scriptsize {\xi}_j \downarrow \, \begin{bmatrix} \, \large e^{- 2 \pi x \cdot {\xi}_j} \, \end{bmatrix} \\ & \end{matrix} [f]_x = \int\limits_{-\infty}^{+\infty} f(x) e^{- 2 \pi x \cdot {\xi}_j} dx \,
\, [f]_{x_k} = \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{\xi} \\ \scriptsize x_k \downarrow \, \begin{bmatrix} \, \large e^{ 2 \pi x_k \cdot \xi} \, \end{bmatrix} \\ & \end{matrix} [f]_{\xi} = \int\limits_{-\infty}^{+\infty} \hat{f}(\xi) e^{2 \pi x_k \cdot \xi} d{\xi} \,
\, M^* = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{{\xi}_k} \\ \scriptsize x_j \downarrow \, \begin{bmatrix} \, \large e^{ -2 \pi x \cdot \xi} \, \end{bmatrix} \\ & \end{matrix} \,
\, M M^* = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{x} \\ \scriptsize {\xi}_j \downarrow \, \begin{bmatrix} \, \large e^{2 \pi x \cdot {\xi}_j} \, \end{bmatrix} \\ & \end{matrix} \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{{\xi}_k} \\ \scriptsize x \downarrow \, \begin{bmatrix} \, \large e^{ -2 \pi x \cdot {\xi}_k} \, \end{bmatrix} \\ & \end{matrix} = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{{\xi}_k} \\ \scriptsize {\xi}_j \downarrow \, \begin{bmatrix} \, \large {\delta}_{{\xi}_j {\xi}_k} \, \end{bmatrix} \\ & \end{matrix} = \;\; I .
\, M^* M = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{\xi} \\ \scriptsize x_j \downarrow \, \begin{bmatrix} \, \large e^{2 \pi x_j \cdot \xi} \, \end{bmatrix} \\ & \end{matrix} \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{x_k} \\ \scriptsize \xi \downarrow \, \begin{bmatrix} \, \large e^{ -2 \pi x_k \cdot \xi} \, \end{bmatrix} \\ & \end{matrix} = \; \begin{matrix} \;\;\;\;\;\; \xrightarrow[]{x_k} \\ \scriptsize x_j \downarrow \, \begin{bmatrix} \, \large {\delta}_{ x_j x_k} \, \end{bmatrix} \\ & \end{matrix} = \;\; I .
///////
Matrix product: \, \int\limits_{-\infty}^{+\infty} e^{2 \pi x \cdot {\xi}_j} e^{ -2 \pi x \cdot {\xi}_k} dx \, = \, \int\limits_{-\infty}^{+\infty} e^{2 \pi x \cdot ({\xi}_j -{\xi}_k)} dx
///////
Convergence theorems:
• Carleson’s theorem
• Statement of the theorem
• Carleson-Hunt theorem
///////
\, \hat{f}(\xi):= \int_{\mathbb{R}} e^{-2\pi i x \cdot \xi} \, f(x) \, dx ,
then
\, f(x)=\int_{\mathbb{R}} e^{2\pi i x \cdot \xi} \, \hat{f}(\xi) \, d\xi .
//////////////////////////////////////
The length of a vector \, x = (x_1, x_2, \cdots, x_n) \, in the \, n -dimensional real vector space \, \mathbb{R}^n is usually given by the Euclidean norm:
\, \left\| x \right\|_2 = \left( {x_1}^2 + {x_2}^2 + \dotsb + {x_n}^2 \right)^{1/2} .
If \, x \in \mathbb{C}^n , its length is instead given by
\, \left\| x \right\|_2 = \left( {|x_1|}^2 + {|x_2|}^2 + \dotsb + {|x_n|}^2 \right)^{1/2} = \left( x_1 \overline{x_1} + x_2 \overline{x_2} + \dotsb + x_n \overline{x_n} \right)^{1/2} .
The \, p -norm in finite dimensions
The Euclidean distance between two points x and y is the length \, \left\| x - y \right\|_2 \, of the straight line between the two points. In many situations, the Euclidean distance is insufficient for capturing the actual distances in a given space. An analogy to this is suggested by taxi drivers in a grid street plane who should measure distance not in terms of the length of the straight line to their destination, but in terms of the rectilinear distance (taxi-cab metric), which takes into account that streets are either orthogonal or parallel to each other. The class of p-norms generalizes these two examples and has an abundance of applications in many parts of mathematics, physics, and computer science.
Definition:
For a real number \, p ≥ 1 , the \, p -norm or \, L^p -norm of \, x \, is defined by
\, \left\| x \right\|_p = \left( |x_1|^p + |x_2|^p + \dotsb + |x_n|^p \right)^{1/p} .
The absolute value bars are unnecessary when \, p \, is a rational number and, in reduced form, has an even numerator.
The Euclidean norm from above falls into this class and is the \, 2 -norm.
The \, 1 -norm is the norm that corresponds to the rectilinear distance (taxicab geometry).
The \, L^{\infty} -norm or maximum norm (or uniform norm) is the limit of the \, L^p -norms for \, p \to \infty . It turns out that this limit is equivalent to the following definition:
\, \left\| x \right\|_{\infty} = \max \left\{ |x_1|, |x_2|, \dotsc, |x_n| \right\} .
See [[L-infinity|{{math|”L”}}-infinity]]
//////////////////////////////////////
\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; LINEAR ALGEBRA IN HILBERT SPACE
\;\;\;\;\;\;\;\; FINITE DIMENSIONS \;\;\;\;\;\;\;\; FOURIER SERIES \;\;\;\;\;\;\;\;\;\;\; FOURIER TRANSFORM
\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; ON \, L^1 \; | \; ON \, L^2 \,
\,\;\;\;\;\;\ Inner product space \;\;\;\; Hilbert space \, ℓ^2({[0,1]}^m) \, \;\;\; Hilbert space \, L^2(\mathbb{R}^m) \,
\,\;\;\;\;\;\;\;\; Orthonormal basis: \;\;\;\;\;\;\; Orthonormal basis: \;\;\;\;\;\;\;\;\; Orthonormal basis:
\;\;\;\;\;\;\, E = \{ e_1, \dots, e_N \} . \;\;\;\;\; F_S = \{ e^{2\pi i x \cdot n} \}_{n \in {\mathbb{Z}}^m} . \;\;\;\;\;\;\;\;\;\, F_T = \{ e^{ 2\pi i x \cdot \xi} \}_{\xi \in {\mathbb{R}}^m} .
\,\;\;\;\;\;\;\;\;\;\;\;\;\; Dual basis: \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; Dual basis: \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; Dual basis:
\;\;\;\;\; E^* = \{ \overline {e_1}, \dots, \overline {e_N} \} . \;\;\;\; {F_S}^* = \{ e^{-2 \pi i x \cdot n} \}_{n \in {\mathbb{Z}}^m} . \;\;\;\;\;\; {F_T}^* = \{ e^{-2\pi i x \cdot \xi} \}_{\xi \in {\mathbb{R}}^m} .
\,\;\;\;\;\;\;\;\;\;\;\; Representation: \;\;\;\;\;\;\;\;\;\; Representation: \,\;\;\;\;\;\;\;\;\;\;\;\;\;\; Representation:
\,\;\;\;\;\;\;\;\;\;\;\; \hat{x} = [x]_E . \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \hat{f} = [f]_{F_S} . \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \hat{f} = [f]_{F_T} .
\,\;\;\;\;\;\;\;\;\;\;\;\; Coordinates: \;\;\;\;\;\;\;\;\;\;\;\;\;\; Coordinates: \,\,\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; Coordinates:
\,\;\;\;\;\;\;\;\;\;\; \hat{x}_n = \left< x , \overline {e_n} \right> \,\;\;\;\; \hat{f}_n = \left< f(x), e^{- 2 \pi i x \cdot n} \right> \;\;\;\;\;\;\;\;\; \hat{f}_\xi = \left< f(x) , e^{- 2\pi i x \cdot \xi} \right>
\;\;\;\;\;\;\;\;\;\;\; = x \cdot \overline {e_n} \, . \;\;\;\;\;\;\;\;\; = \int\limits_{x \in {[0,1]}^m} f(x) e^{- 2 \pi i x \cdot n} dx . \;\;\;\;\; = \int\limits_{x \in {\mathbb{R}}^m} f(x) e^{-2\pi i x \cdot \xi} dx .
\,\;\;\;\;\;\;\;\;\;\; Reconstruction: \;\;\;\;\;\;\;\;\;\; Reconstruction: \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; Reconstruction:
\;\;\;\;\;\;\;\;\;\;\; x = (x)_E = \;\;\;\;\;\;\;\;\;\;\; f(x) = (\hat{f})_{F_S} = \,\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; f(x) = (\hat{f})_{F_T} =
\,\;\;\;\;\;\;\;\;\;\; = \sum\limits_{n=1}^{N} \hat{x}_n e_n . \,\;\;\;\;\;\;\;\;\;\;\; = \sum\limits_{n \in {\mathbb{Z}}^m} \hat{f}_n e^{2 \pi i x \cdot n} . \,\;\;\;\;\;\;\;\;\;\;\;\;\;\; = \int\limits_{\xi \in {\mathbb{R}}^m} \hat{f}_{\xi} e^{ 2\pi i x \cdot \xi} d\xi .
\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; Analysis and Synthesis \;\;\;\;\;\;\;\;\;\;\;\;\;\; Applications
///////
But what is a Fourier Series? – From heat flow to circle drawings
(3Blue1Brown on YouTube):
///////
Parseval’s theorem:
Quoting Wikipedia: (on Parseval’s theorem)
In mathematics, Parseval’s theorem usually refers to the result that the Fourier transform is unitary; loosely, that the sum (or integral) of the square of a function is equal to the sum (or integral) of the square of its transform. It originates from a 1799 theorem about series by Marc-Antoine Parseval, which was later applied to the Fourier series. It is also known as Rayleigh’s energy theorem, or Rayleigh’s identity, after John William Strutt, Lord Rayleigh.
Although the term “Parseval’s theorem” is often used to describe the unitarity of any Fourier transform, especially in physics, the most general form of this property is more properly called the Plancherel theorem.
///////
Statement of Parseval’s theorem: (Quoting Wikipedia):
Suppose that \, A(x) \, and \, B(x) \, are two complex-valued functions on \, \mathbb{R}\, of period \, 2\pi\, that are square integrable (with respect to the Lebesgue measure) over intervals of period length, with Fourier series
\, A(x)=\sum\limits_{n=-\infty}^{\infty} a_n e^{inx} \,
and
\, B(x)=\sum\limits_{n=-\infty}^{\infty} b_n e^{inx} \,
respectively. Then
\, \sum\limits_{n=-\infty}^{\infty} a_n \overline{b_n} = \frac{1}{2\pi} \int\limits_{-\pi}^{\pi} A(x)\overline{B(x)} \, \mathrm{d}x \,
where \, i \, is the imaginary unit and horizontal bars indicate complex conjugation. Substituting \, A(x) \, and \, \overline{B(x)} \, :
\, \sum\limits_{n=-\infty}^{\infty} a_n \overline{b_n} = \,
\, = \frac{1}{2\pi} \int\limits_{-\pi}^{\pi} \left( \sum\limits_{n=-\infty}^{\infty} a_n e^{inx} \right) \left( \sum\limits_{n=-\infty}^{\infty}\overline{b_n} e^{-inx} \right) \mathrm{d}x = \,
\, = \frac{1}{2\pi} \int\limits_{-\pi}^\pi (a_1e^{i1x} + a_2e^{i2x} + ...) (\overline{b_1}e^{-i1x} + \overline{b_2}e^{-i2x} + ...) \, \mathrm{d}x = \,
\, = \frac{1}{2\pi} \int\limits_{-\pi}^\pi (a_1e^{i1x} \overline{b_1}e^{-i1x} + a_1e^{i1x} \overline{b_2} e^{-i2x} + a_2e^{i2x} \overline{b_1}e^{-i1x} + ...) \, \mathrm{d}x = \,
\, = \frac{1}{2\pi} \int\limits_{-\pi}^{\pi} (a_1 \overline{b_1} + a_1 \overline{b_2} e^{-ix} + a_2 \overline{b_1} e^{ix} + a_2 \overline{b_2} + ...) \, \mathrm{d}x .
As is the case with the middle terms in this example, many terms will integrate to \, 0 \, over a full [[Periodic_function|period]] of length \, 2\pi \, (see [[harmonic|harmonics]]):
\, \sum\limits_{n=-\infty}^{\infty} a_n\overline{b_n} = \,
\, = \frac{1}{2\pi} (a_1 \overline{b_1} x + i a_1 \overline{b_2} e^{-ix} - i a_2 \overline{b_1} e^{ix} + a_2 \overline{b_2} x + ...)\vert_{-\pi} ^{+\pi} = \,
\, = \frac{1}{2\pi} (2\pi a_1 \overline{b_1} + 0 + 0 + 2\pi a_2 \overline{b_2} + ...) = a_1 \overline{b_1} + a_2 \overline{b_2} + ... \, .
///////
///////
How to show that Plancherel’s Theorem for the Fourier Transform implies 𝐿2 Transform Convergence.
Quotations from this source:
///
Background: This is not a homework problem or something I found in a text. It would seem reasonable to expect this result because of the discrete version where Parseval’s equality for the Fourier series implies \, L^2 \, convergence of the Fourier series; or, if { \, \{ \,e_n \, \} }_{n=1}^{\infty} \, is an orthonormal set in an inner product space, then
///
///
Showing that the Fourier series converges in norm if Parseval’s identity holds, is straightforward. In fact, it’s a general Hilbert space property that norm convergence is implied by Parseval’s identity for the Fourier series. It should be possible to use the Parseval identity for the Fourier transform to prove \, L^2 \, norm convergence of the inverse Fourier transform of the Fourier transform (which would not imply pointwise convergence.)
///
///////
Plancherel’s theorem:
Quoting Wikipedia: (on Plancherel’s theorem):
In mathematics, the Plancherel theorem (sometimes called the Parseval–Plancherel identity) is a result in harmonic analysis, proven by Michel Plancherel in 1910. It states that the integral of a function’s squared modulus is equal to the integral of the squared modulus of its frequency spectrum. That is, if \, f(x) \, is a function on the real line, and \, \widehat {f}(\xi) \, is its frequency spectrum, then
\, \int\limits_{-\infty}^{\infty} {|f(x)|}^2 dx \, = \, \int\limits_{-\infty}^{\infty} {|\widehat{f}(\xi)|}^2 d\xi .
A more precise formulation is that if a function is in both \, L^p \, spaces \, L^1({\mathbb {R}}) \, and \, L^2(\mathbb{R}) , then its Fourier transform is in \, L^2(\mathbb{R}) , and the Fourier transform map is an isometry with respect to the \, L^2 \, norm.
This implies that the Fourier transform map restricted to \, L^1(\mathbb {R}) \cap L^2(\mathbb {R}) \, has a unique extension to a linear isometric map \, L^{2}(\mathbb {R})\rightarrow L^{2}(\mathbb {R}) , sometimes called the Plancherel transform. This isometry is actually a unitary map. In effect, this makes it possible to speak of Fourier transforms of quadratically integrable functions.
Plancherel’s theorem remains valid as stated on n-dimensional Euclidean space \, \mathbb {R}^n .
The unitarity of the Fourier transform is often called Parseval’s theorem in science and engineering fields, based on an earlier (but less general) result that was used to prove the unitarity of the Fourier series.
Due to the polarization identity, one can also apply Plancherel’s theorem to the \, L^2(\mathbb{R}) \, inner product of two functions. That is, if \, f(x) \, and \, g(x) \, are two \, L^2(\mathbb{R}) \, functions, and \, \mathcal {P} \, denotes the Plancherel transform, then
\,\int\limits_{-\infty}^\infty f(x)\overline{g(x)} \, dx \, = \, \int\limits_{-\infty}^\infty (\mathcal P f)(\xi) \overline{(\mathcal P g)(\xi)} \, d\xi ,
and if \, f(x) \, and \, g(x) \, are furthermore \, L^1(\mathbb{R}) \, functions, then
\, (\mathcal P f)(\xi) = \widehat{f}(\xi) = \int\limits_{-\infty}^\infty f(x) e^{-2\pi i \xi x} \, dx ,
and
\, (\mathcal P g)(\xi) = \widehat{g}(\xi) = \int\limits_{-\infty}^{\infty} g(x) e^{-2\pi i \xi x} \, dx ,
so
\, \int\limits_{-\infty}^{\infty} f(x)\overline{g(x)} \, dx = \int\limits_{-\infty}^{\infty} \widehat{f}(\xi) \overline{\widehat{g}(\xi)} \, d\xi .