Some Math

Category: Real Analysis

Cauchy’s criterion on almost sure convergence

In this article, we will investigate interplays between various kinds of convergence of sequence of random variables. Specifically, we will discuss convergence a.s., convergence in probability, convergence in {p}th mean and convergence in distribution.

We can naturally formalize the notions of sequence of random variables being Cauchy in these contexts of convergence. We say: {X_n} is Cauchy in probability if {P(|X_m-X_n|>\epsilon)\rightarrow 0} as {m, n\rightarrow\infty}; {X_n} is Cauchy a.s. if {X_n} is Cauchy everywhere except a set of measure zero; {X_n} is Cauchy in {L^p} if {\mathop{\mathbb E} |X_n-X_m|^p\rightarrow 0} as {m,n\rightarrow\infty}. Now we give some basic facts about the advantages of using Cauchy’s criterion.

Proposition 1 {X_n\rightarrow X} a.s. if and only if {P(\sup_{k\geq n}|X_k-X|\geq\epsilon)\rightarrow 0} for every {\epsilon>0} as {n\rightarrow\infty}.

Proof: Let us consider the formulation of the set of points where {X_n\not\rightarrow X}; that is

\displaystyle \{\omega: X_n(\omega)\not\rightarrow X(\omega)\}=\cup_{\epsilon>0}\cap_{n=1}^\infty\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq\epsilon\}

In fact, we can use the denseness of real numbers to write

\displaystyle \{\omega: X_n(\omega)\not\rightarrow X(\omega)\} =\cup_{j=1}^\infty\cap_{n=1}^\infty\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq1/j\}


\displaystyle \begin{array}{rcl} X_n\xrightarrow{a.s.} X & \Leftrightarrow P(\{\omega: X_n(\omega)\not\rightarrow X(\omega)\}=0\\ & \Leftrightarrow P\left(\cup_{j=1}^\infty\cap_{n=1}^\infty\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq1/j\}\right)=0\\ & \Leftrightarrow P\left(\cap_{n=1}^\infty\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq1/j\}\right)=0\;\;\forall j\in{\mathbb N}\\ & \Leftrightarrow P\left(\cap_{n=1}^\infty\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq\epsilon\}\right)=0\;\;\forall \epsilon\in{\mathbb R}^+\\ & \Leftrightarrow P\left(\cup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq\epsilon\}\right)\rightarrow 0\;\;\forall \epsilon\in{\mathbb R}^+\\ & \Leftrightarrow P\left(\sup_{k\geq n}\{\omega:|X_k(\omega)-X_n(\omega)|\geq\epsilon\}\right)\rightarrow 0\;\;\forall \epsilon\in{\mathbb R}^+ \end{array}


Proposition 2 {X_n} is Cauchy a.s. if and only if {P(\sup_{\substack{k\geq n\\l\geq n}}|X_k-X_l|\geq \epsilon)\rightarrow 0} as {n\rightarrow\infty} for every {\epsilon>0}; or equivalently {P(\sup_{k\geq 0}|X_{n+k}-X_n|\geq\epsilon)\rightarrow 0} as {n\rightarrow\infty} for every {\epsilon>0}.

Proof: Using a similar argument, we obtain a chain of equivalences,

\displaystyle \begin{array}{rcl} X_n \text{ is Cauchy a.s. }&\Leftrightarrow P(\{\omega: X_n \text{ is not Cauchy}\})=0\\ & \Leftrightarrow P\left(\cup_{j=1}^\infty\cap_{n=1}^\infty\cup_{\substack{k\geq n\\l\geq n}}\{\omega:|X_k-X_l|\geq 1/j\}\right)=0\\ & \Leftrightarrow P\left(\cap_{n=1}^\infty\cup_{\substack{k\geq n\\l\geq n}}\{\omega:|X_k-X_l|\geq 1/j\}\right)=0\;\;\forall j\in{\mathbb N}\\ & \Leftrightarrow P\left(\cap_{n=1}^\infty\cup_{\substack{k\geq n\\l\geq n}}\{\omega:|X_k-X_l|\geq \epsilon\}\right)=0\;\;\forall \epsilon\in{\mathbb R}^+\\ & \Leftrightarrow P\left(\cup_{\substack{k\geq n\\l\geq n}}\{\omega:|X_k-X_l|\geq \epsilon\}\right)\rightarrow 0\;\;\forall \epsilon\in{\mathbb R}^+\\ & \Leftrightarrow P\left(\sup_{\substack{k\geq n\\l\geq n}}\{\omega:|X_k-X_l|\geq \epsilon\}\right)\rightarrow 0\;\;\forall \epsilon\in{\mathbb R}^+\\ & \Leftrightarrow P\left(\sup_{\substack{k\geq 0}}\{\omega:|X_{n+k}-X_n|\geq \epsilon\}\right)\rightarrow 0\;\;\forall \epsilon\in{\mathbb R}^+ \end{array}

where the last equivalence is obtain from the following inequality

\displaystyle \sup_{k\geq 0}|X_{n+k}-X_n|\leq\sup_{\substack{k\geq 0\\l\geq 0}}|X_{n+k}-X_{n+l}|\leq 2\sup_{k\geq 0}|X_{n+k}-X_n|

This completes the proof. \Box

Remark: Similarly one can formulate another equivalent definition of Cauchy a.s., that is if {P(\sup_{m<k\leq n}|X_k-X_m|>\epsilon)\rightarrow 0} for all {\epsilon>0} as {n,m\rightarrow\infty}, then {X_n} is Cauchy a.s. Let {\eta>0}. Then there is an integer {N, M} such that {P(\sup_{m<k\leq n}|X_k-S_m|>\epsilon)<\eta} whenever {n>N, m>M}. Sending {N\rightarrow\infty} gives {P(\sup_{k>m}|X_k-X_m|>\epsilon)<\eta} which is just the previous proposition.


Definition: If there exists a 1-1 mapping of {A} onto {B}, we say that {A} and {B} can be put in 1-1 correspondence , or that {A} and {B} have the same cardinal number , or briefly, that {A} and {B} are equivalent, and we write {A\sim B}. This relation clearly has the following three properties:

  1. It is reflexive: {A\sim A}.
  2. It is symmetric: if {A\sim B}, then {B\sim A}.
  3. It is transitive: if {A\sim B} and {B\sim C}, then {A\sim C}.

Definition: For any positive integer {n}, let {J_{n}} be the set whose elements are the integers, {1,2,...,n}; let {J} be the set of all positive integers. For any set {A}, we say:

  1. (a) {A} is finite if {A\sim J_{n}} for some positive integer {n}.
  2. (b) {A} is infinite if {A} is not finite.
  3. (c) {A} is countable if {A\sim J}.
  4. (d) {A} is uncountable if {A} is neither finite nor countable.
  5. (d) {A} is at most countable if {A} is finite or countable.

Remark: A finite set cannot be equivalent to one of its proper subsets. However, this is possible for infinite sets. In fact, we could restate the definition (b): {A} is infinite if {A\sim B} for some {B\subset A}.

Theorem 1 Every infinite subset of a countable set {A} is countable.

Proof: Suppose {B\subseteq A} and {B} is infinite. Arrange the elements of {A} which are all distinct in a sequence {(a_{i})_{i=1}^{\infty}}. Then we construct a sequence {(n_{k})_{k=1}^{\infty}} as follows:

Let {n_{1}\in J} be the smallest number such that {a_{n_{1}}\in B}. Note {n_{1}} exists because {\mathbb{N}} is well-ordered. After choosing {n_{1},...,n_{k-1}}, we let {n_{k}} be the smallest number greater than {n_{k-1}} such that {a_{n_{k}}\in B}.

Then consider the function {f: J\rightarrow B} where {f(k)=a_{n_{k}}}. We then check {f} is indeed a bijection. It is one-to-one because {(a_{i})_{i=1}^{\infty}} is a distinct sequence; then for any {i\neq j}, we have {a_{n_{i}}\neq a_{n_{j}}}. {f} must be surjective for otherwise, there exists {b\in B} is not in the image of {(a_{i})_{i=1}^{\infty}} contradicting the fact that {B\subseteq A}.


Corollary 2 Every subset of an at most countable set is at most countable.

Proof: Let {A} be an at most countable set and {B\subseteq A}. If {A} is finite, then {B} is must also be finite. So let us assume {A} is countable. Then {B} cannot be uncountable by theorem 1.


Corollary 3 An at most countable set that contains a countable subset is countable.

Proof: Let {A} be at most countable and {B\subseteq A} is countable. If {A} is countable then we are done. So we need to prove that {A} cannot be finite. Suppose {A} is finite then there exists a bijection {f: A\rightarrow J_{n}}. Let {g: J\rightarrow B}. Then the composition {f\circ g:\mathbb{N}\rightarrow J_{n}} must be injective. However, by pigeonhole’s principle we know this cannot happen, thus a contradiction.


Theorem 4 Countable unions of countable sets is countable.

Corollary 5 At most countable unions of at most countable sets is at most countable.

Let {\left\{X_{\alpha}|\alpha\in A\right\}} be an arbitrary family of sets. We define the Cartesian product {\prod_{\alpha\in A}X_{\alpha}} to be all the functions {\phi: A\rightarrow\bigcup_{\alpha\in A}X_{\alpha}} such that {\phi(\alpha)\in X_{\alpha}}. We can think of {\phi(\alpha)} as an evaluation of {\phi} at the {\alpha}-th entry. For example if {A=\mathbb{N}}, {\phi(1)} is simply the first entry of the Cartesian product.

Theorem 6 Finitely many Cartesian products of countable sets is countable.

Theorem 7 Countable Cartesian products of countable sets is uncountable.

Proof: Let {S:=\prod_{n=1}^{\infty} A_{n}} where {A_{n}} is countable for each {n\in J}. Let {(f_{i}: i\in J)} be an enumeration. For each {n\in J} we pick two different element {a_{n}, b_{n}\in A_{n}}. Consider the following function:

\displaystyle f(j) = \left\{ \begin{array}{lr} b_{j} & : \mathrm{if}\, f_{j}(j)=a_{j}\\ a_{j} & : \mathrm{otherwise} \end{array} \right.

Note {f\in S} obviously. However, note {f\neq f_{i}} for all {i\in J} because {f} differs from every {f_{i}} at least at {f_{j}(j)}. Thus {S} in not countable.


Corollary 8 Let {A} be the set of all sequences whose elements are the digis 0 and 1. Then {A} is uncountable.

Proof: Remember the set {A} is indeed identified with the Cartisian product {\left\{0,1\right\}^{\mathbb{N}}}. Although we cannot directly apply theorem 7 because {\left\{0,1\right\}} is finite, we still can use the same Cantor trick. Let {(f_{i}: i\in J)} be an enumeration of {A}. We define the following function

\displaystyle f(j) = \left\{ \begin{array}{lr} 0 & : \mathrm{if}\, f_{j}(j)=1\\ 1 & : \mathrm{otherwise} \end{array} \right.

Again {f\in A} obviously, but {f} differs from {f_{i}} for every {i\in J}. Thus {A} is uncountable.


Theorem 9 There exists no surjection from a set {X} to {\mathcal{P}(X)}.

Proof: We prove by contradiction. Let {\varphi: X\rightarrow\mathcal{P}(X)} be a surjection. By definition, for any {A\in\mathcal{P}(X)} there exists {a\in X} such that {\varphi(x)=A}. Consider the following set

\displaystyle Y:=\left\{x\in X| x\notin\varphi(x)\right\}

So {Y} has an pre-image in {X}. Let {y\in X} such that {\varphi(y)=Y}. It is either {y\in Y} or {y\notin Y}. If it is the first case, then {y\in X} and {y\notin\varphi(y)=Y} by definition, but his contradicts the fact that {y\in Y}. If it is the second case, we have {y\notin Y}, so {y\in X} and {y\in\varphi(y)=Y}, contradiction again. Hence {\varphi} cannot be a surjection and the proof is complete.


Littlewood’s three principles (3)

Finally the principle states that every convergent sequence nearly converges uniformly. Let’s figure out what the word “nearly” means in this context and this brings us to the Egorov’s theorem.


Theorem 1 (Egorov)Suppose {\left\{f_{n}\right\}_{i=1}^{\infty}} is a sequence of measurable functions defined on a measurable set {E} with {m(E)<\infty}, and assume {f_{i}\rightarrow f} almost everywhere on {E}. Given {\epsilon>0}, we can find a closed set {A_{\epsilon}\subseteq E} such that {m(E-A_{\epsilon})\leq\epsilon} and {f_{i}\rightarrow f} uniformly on {E}.

Proof: We may assume {f_{i}\rightarrow f} for every {x\in E} since we can always find a subset {E'\subseteq E} which differs with {E} by a measure zero set such that the convergence holds everywhere on {E'}, then we just rename {E'} by {E}. Define

\displaystyle E_{k}^{n}:=\left\{x\in E:\left|f_{j}(x)-f(x)\right|<\frac{1}{n}\;\forall j>k\right\}

Note for fixed {n} we have the relation {E_{k}^{n}\subseteq E_{k+1}^{n}} and {E_{k}^{n}\nearrow E} as {k\rightarrow\infty}. Then by the continuity of measures we know {\lim_{k\to \infty}m(E_{k}^{n})=m(E)} which implies we can find an integer {k_{n}\in\mathbb{N}} such that {m(K-E_{k_{n}}^{n})<\frac{1}{2^{n}}}. By construction, we have {\left|f_{j}(x)-f(x)\right|<\frac{1}{n}} whenever {j>k_{n}} and {x\in E_{k}^{n}}. Then we choose {N\in\mathbb{N}} so that {\sum_{n=N}^{\infty}<\frac{\epsilon}{2}} and let {\hat{A}_{\epsilon}:=\bigcap_{n\geq N}E_{k}^{k_{n}}}.
First we notice the relation {m(E-\hat{A}_{\epsilon})=m(\bigcup_{n\geq N}E-E_{k}^{k_{n}})\leq\sum_{n=N}^{\infty}(E-E_{k}^{k_{n}}<\frac{\epsilon}{2})}. Next for any {\delta>0}, we can choose {N'\in\mathbb{N}} such that {\frac{1}{n}<\delta} when {n>N'}. Now we indeed have completed our job as for any {\delta>0} we can find an integer {k_{n}} such that for all {j>k_{n}} we \left|f_{j}(x)-f(x)\right|<\delta<\frac{1}{n} for all {x\in\hat{A}_{\epsilon}}.

Finally, as a technical requirement, we can find a closed subset {A_{\epsilon}\subseteq\hat{A}_{\epsilon}} such that {m(\hat{A}_{\epsilon}-A_{\epsilon})<\frac{\epsilon}{2}} which leads us to the desired property {m(E-A_{\epsilon})<\epsilon}.                                                                                                                                                                           \Box

Littlewood’s three principles (2)

The second principle asserts that every function is nearly continuous. Now we investigate the meaning of “nearly” in terms of mathematics.

Theorem 2 (Lusin) Suppose f is measurable and finite valued on E with E of finite measure. Then for every {\epsilon>0} there exists a closed set {F_{\epsilon}} with

\displaystyle F_{\epsilon}\subseteq E\;\mathrm{and}\;m(E-F_{\epsilon})\leq\epsilon

and such that {f|_{F_{\epsilon}}} is continuous.

Proof: Since by assumption, {f} is measurable and finite valued on E, we can find a sequence of step functions, {f_{n}} such that {f_{n}\rightarrow f} almost everywhere. Note that each {f_{n}} is a step function which means it is continuous almost everywhere as it contains only at most countably many discontinuities which have Lebesgue measure 0. So without loss of generality, say for each {n} we can find sets {E_{n}} with {m(E_{n})\leq\frac{1}{2^{n}}} so that {f_{n}} is continuous outside {E_{n}}. Now to proceed, we need something extra which is obtained from the third littlewood’s principle, i.e., Egrovff’s theorem. It enables us to find a set {A_{\frac{\epsilon}{3}}} with {m(E-A_{\frac{\epsilon}{3}})\leq\frac{\epsilon}{3}} such that {f_{n}\rightarrow f} uniformly on {A_{\frac{\epsilon}{3}}}. Then for the integer N such that {\sum_{n\geq N}\frac{1}{2^{n}}\leq\frac{\epsilon}{3}}, we consider

\displaystyle F':=A_{\frac{\epsilon}{3}}-\bigcup_{n\geq N} E_{n}

For every {n\geq N}, the step function {f_{n}} is continuous on {F'}; therefore, the uniform limit {f} is also continous on {F'}. To complete the proof, notice that {F'} is also measurable so we can approximate from below by a closed set {F_{\epsilon}} such that {m(F'-F_{\epsilon})\leq\frac{\epsilon}{3}} which gives us the desired {m(E-F_{\epsilon})\leq\epsilon}.

A Mild Glimpse Into Measure Theory (2)

3. Measures and Outer measures


Let {(E,\mathscr{M})} be a measurable space.

Definition: A measure on {(E,\mathscr{M})} is a function {\mu:\mathscr{M}\rightarrow [0,\infty]} such that:
(1) {\mu(\emptyset)=0}
(2) For all families {\left\{A_{n}\right\}_{n\in\mathbb{N}}} of disjoint measurable sets,

\displaystyle \mu(\bigcup_{n\in\mathbb{N}} A_{n})=\sum_{n\in\mathbb{N}}\mu(A_{n})


  • {\mu(A)=+\infty} is allowed
  • Property (2) is called {\sigma}-additivity
  • We can also derive {\mu(\bigcup_{n=1}^{k} A_{n})=\sum_{n=1}^{k}\mu(A_{n})}


Proposition 6 : Let {A,B\in\mathscr{M}}, we have the following:
(1) If {A\subseteq B}, then {\mu(A)\leq\mu(B)}.
(2) If {\mu(A)<+\infty} and {A\subseteq B}, then {\mu(A\setminus B)=\mu(A)-\mu(B)}
(3) {\mu(A\cup B)=\mu(A)+\mu(B)-\mu(A\cap B)}
(4) If {A_{n}\in\mathscr{M}} for all {n\in\mathbb{N}} and {A_{n}\subseteq A_{n+1}} for all {n\in\mathbb{N}}, then {\mu(\bigcup_{n\in\mathbb{N}}A_{n})=\lim_{n\rightarrow\infty}\mu(A_{n})}.
(5) If {B_{n}\in\mathscr{M}} for all {n\in\mathbb{N}} and {B_{n}\supseteq B_{n+1}} for all {n\in\mathbb{N}}, then {\mu(\bigcap_{n\in\mathbb{N}}B_{n})=\lim_{n\rightarrow\infty}\mu(B_{n})}.
(6) If {A_{n}\in\mathscr{M}}, then {\mu(\bigcup_{n\in\mathbb{N}}A_{n})\leq\sum_{n\in\mathbb{N}}A_{n}}.

To be updated.


Littlewood’s three principles (1)

Littlewood principles describe the relationship between concepts in measure space and concepts in topological space. He summerized these connections which provide a quite intuitive guide:

  • Every set is nearly a finite union of intervals.
  • Every function is nearly continuous.
  • EVery convergent sequence is nearly uniformly convergent.

However, in mathematics we need a precise meaning of the word “nearly”. So let us investigate them in each case one by one. First off, let us introduce the notation we will use. {m(E)} denotes Lebesgue measure of a measurable subset {E} in {\mathbb{R}^{d}}, where we define Lebesgue measurable in the sense that for every {\epsilon}, there exists an open set {O} such that {E\subseteq O} and {m_{*}(O-E)\leq\epsilon} and for Lebesgue measurable subset {E}, {m(E)=m_{*}(E)}. Here {m_{*}(E)} denotes the Lebesgue outer measure of a measurable subset which is defined to be {\mathrm{inf}\sum_{i=1}^{\infty}\left|Q_{i}\right|} and the infimum is taken over all countable coverings of {E} by closed cubes.

Theorem 1 Suppoe E is a measurable subset of {\mathbb{R}^{d}} with finite measure. Then, for every {\epsilon>0}, there exists a finite union {F=\bigcup_{i=1}^{N}Q_{i}} of closed cubes such that {m(E\triangle F)\leq\epsilon}.

Proof: Fix {\epsilon>0}. By definition, for this {\epsilon}, we are able to choose a family of closed cubes {\left\{Q_{i}\right\}_{i=1}^{\infty}} such that {E\subseteq\bigcup_{i=1}^{\infty}Q_{i}} and {\sum_{i=1}^{\infty}\left|Q_{i}\right|\leq m(E)+\epsilon}. Since we assume {m(E)<\infty}, the series on the left hand side converges, i.e., there exists {N>0} such that {\sum_{i>N}\left|Q_{i}\right|<\epsilon}. Now we let {F=\bigcup_{i=1}^{N}Q_{i}}, then

\displaystyle \begin{array}{lll} m(E\triangle F) & = & m(E-F)+m(F-E)\\ & \leq & m(\bigcup_{i=N+1}^{\infty}Q_{i})+m(\bigcup_{i=1}^{\infty}Q_{i}-E)\\ & \leq & \sum_{i>N}\left|Q_{i}\right|+\sum_{i=1}^{\infty}\left|Q_{i}\right|-m(E)\\ &\leq& 2\epsilon \end{array}



A Mild Glimpse Into Measure Theory (1)

1. Introduction

The key idea is to define a function which assigns a value {m(A)}, meaning “the size of {A}”, to a subset {A} of a given set {E}, so that

\displaystyle m(A\cup B)=m(A)+m(B)\;\mathrm{if}\;A\cap B=\emptyset

e.g. number of elements, length of an interval.

However, some issues arise in general since one cannot define a function {m:\mathscr{P}(E)\longrightarrow[0,\infty]} which satisfies above or more generally

\displaystyle m(\bigcup_{n\in\mathbb{N}}A_{n})=\sum_{n\in\mathbb{N}}m(A_{n})\;\; \mathrm{\;if\;\left\{A_{n}\right\}_{n\in\mathbb{N}}}\mathrm{\;is\;a\;family\;of\;disjoint\;sets}

Therefore, we should restrict the domain to be a special class of subsets, so called the {\sigma}-algebra.

2. Measureable Sets


2.1. Definition and Examples

Definition: Let {X} be an arbitrary set. An algebra, {\mathscr{M}\subseteq\mathscr{P}(X)}, is a family of subsets of {X} such that:

  1. {X\in\mathscr{M}}
  2. If {A\in\mathscr{M}, \mathrm{then}\;A^{c}\in\mathscr{M}}
  3. If {A_{n}\in\mathscr{M}} for all {n} from {1} to {N}, then {\bigcup_{n=1}^{N}A_{n}\in\mathscr{M}}
  4. If {A_{n}\in\mathscr{M}} for all {n} from {1} to {N}, then {\bigcap_{n=1}^{N}A_{n}\in\mathscr{M}}

Definition: Let {X} be an arbitrary set. A {\sigma}algebra, {\mathscr{M}\subseteq\mathscr{P}(X)}, is a family of subsets of {X} such that:

  1. {X\in\mathscr{M}}
  2. If {A\in\mathscr{M}, \mathrm{then}\;A^{c}\in\mathscr{M}}
  3. If {A_{n}\in\mathscr{M}} for all {n\in\mathbb{N}}, then {\bigcup_{n\in\mathbb{N}}A_{n}\in\mathscr{M}}
  4. If {A_{n}\in\mathscr{M}} for all {n\in\mathbb{N}}, then {\bigcap_{n\in\mathbb{N}}A_{n}\in\mathscr{M}}

We call elements {M\in\mathscr{M}} a {\mathscr{M}}measureable set, and call the pair {(X,\mathscr{M})}measurable space.

Remark 1: Note we can omit (3) or (4) in each definition by de Morgan laws and we can also replace condition (1) by restricting {\mathscr{M}} to be a nonempty family of subsets of {X}.
Remark 2: From this definition, we can see also: {(a)} {\emptyset\in\mathscr{M}}; {(b)} {\bigcap_{n\in\mathbb{N}}^{k} A_{n}\in\mathscr{M}} and {\bigcup_{n\in\mathbb{N}}^{k} A_{n}\in\mathscr{M}} for any finite integer {k}; hence a {\sigma}-algebra is an algebra but not vice versa.


  • {\mathscr{M}=\mathscr{P}(X)} is a {\sigma}-algebra.
  • {\mathscr{M}=\left\{\emptyset, X\right\}} is a {\sigma}-algebra on {X}.
  • {\mathscr{M}=\left\{A\in\mathscr{P}(X)|A\;\mathrm{or}\; A^c\;\mathrm{is}\;\mathrm{at}\;\mathrm{most}\;\mathrm{countable}\right\}} is a {\sigma}-algebra.
  • Let {X} be an infinite set, and let {\mathscr{F}} be the collection of all finite subsets of {X}. Then {\mathscr{F}} is not even an algebra since {X\notin\mathscr{F}}.
  • Let {X} be an infinite set, and let {\mathscr{F}} be the collection of all subsets {F} of {X} such that either {F} or {F^{c}} is finite. Then {\mathscr{F}} is an algebra but not a {\sigma}-algebra.

We now consider ways of constructing {\sigma}-algebras, first starting with a proposition.

Proposition 1 Let X be an arbitrary set, and let {\left\{\mathscr{M}_{j}\right\}_{j\in J}} be a nonempty collection of {\sigma}-algebras on {X}. Then {\bigcap_{j\in J}\mathscr{M}_{j}} is a {\sigma}-algebra on X.

Proof: Let {\mathscr{A}=\bigcap_{j\in J}\mathscr{M}_{j}}. We need to check that {\mathscr{A}} fulfills the definiton. First, note that {X\in\mathscr{M}_{j}} for all {j} since each {\mathscr{M}_{j}} is a {\sigma}-algebra, so {X\in\mathscr{A}}. Then if {A\in\mathscr{A}}, {A\in\mathscr{M}_{j}} for all {j} by definition, thereby {A^{c}\in\mathscr{M}_{j}} for all {j}, which implies {A^{c}\in\mathscr{A}}. Lastly, let {\left\{A_{n}\right\}_{n\in\mathbb{N}}} be a sequence of elements of {\mathscr{A}}, then for all {n\in\mathbb{N}} we have {A_{n}\in\mathscr{M}_{j}} for all {j\in J}. So we know {\bigcup_{n\in\mathbb{N}}\in\mathscr{M}_{j}} for all {j} by definition of {\sigma}-algebra, hence {\bigcup_{n\in\mathbb{N}}\in\mathscr{A}}.


Corollary 2 Let X be an arbitrary set, and let {\mathscr{F}} be a family of subsets of X. Then there exists a smallest {\sigma}-algebra on X that includes {\mathscr{F}}.

Proof: Let {\mathscr{C}} be the collection of {\sigma}-algebras that includes {\mathscr{F}} and note {\mathscr{C}\neq\emptyset} since {\mathscr{P}(E)\in\mathscr{C}}. Then by proposition 1, the intersection of {\sigma}-algebras in {\mathscr{C}} is desired.


We have this basic tool to construct {\sigma}-algebras. Consider any family of subsets of {E}, {\mathscr{C}\subseteq\mathscr{P}(E)}. Then there exist a unique smallest {\sigma}-algebra {\sigma(\mathscr{C})\supseteq\mathscr{C}}, i.e., {\forall C\in\mathscr{C}\;\;C\in\sigma(\mathscr{C})}. We call {\sigma(\mathscr{C})} the {\sigma}-algebra generated by {\mathscr{C}}. Namely, any {\sigma}-algebra containing {\mathscr{C}} also contains {\sigma(\mathscr{C})}. Now we turn our attention to an important {\sigma}-algebras.

Remark3: By corollary 2, we have an important relationship between {\sigma}-algebra and its subfamilies. If {\mathscr{M}} is a {\sigma}-algebra and {\mathscr{A}\subseteq\mathscr{M}}, then {\sigma(\mathscr{A})\subseteq\mathscr{M}} which will be used in the proof of the following proposition.

Definition: The Borel {\sigma}-algebra on {\mathbb{R}^{n}} is the {\sigma}-algebra generated by the collection/family of open subsets of {\mathbb{R}^{n}}, denoted by {\mathscr{B}(\mathbb{R}^{n})}. We call elements of {\mathscr{B}(\mathbb{R}^{n})} the Borel subsets of {\mathbb{R}^{n}}. In particular, if {n=1}, one often writes {\mathscr{B}(\mathbb{R})}.


Proposition 3 {\mathscr{B}(\mathbb{R})} is generated by each of the following collections of sets:

  1. the collection of all closed subsets of {\mathbb{R}}.
  2. the collection of all subintervals of {\mathbb{R}} of the form {(-\infty, b]}.
  3. the collection of all subintervals of {\mathbb{R}} of the form {(a, b]}.


Proof: Let’s denote the corresponding {\sigma}-algebra generated by the family of sets in (1), (2), (3) by {\mathscr{B}_{1}}, {\mathscr{B}_{2}}, {\mathscr{B}_{3}} respectively. It then suffices to show {\mathscr{B}(\mathbb{R})\supseteq\mathscr{B}_{1}\supseteq\mathscr{B}_{2}\supseteq\mathscr{B}_{3}} and {\mathscr{B}_{3}\supseteq\mathscr{B}(\mathbb{R})}. First consider {\mathscr{C}}, the collection of all closed subsets of {\mathbb{R}}. Notice each any closed subset is just the complement of open subsets so every closed subsets of {\mathbb{R}} is an element of the {\sigma}-algebra {\mathscr{B}(\mathbb{R})}, i.e., {\mathscr{C}\subseteq\mathscr{B}(\mathbb{R})} and by proposition 2 we know {\mathscr{B}_{1}=\sigma(\mathscr{C})\subseteq\mathscr{B}(\mathbb{R})}. Secondly, since each subintervals of the form {(-\infty,b]} is closed we have {\mathscr{B}_{2}(\mathbb{R})\subseteq\mathscr{B}_{1}(\mathbb{R})}. Finally, intervals of the form {(a,b]} can be written as

\displaystyle (a,b]=(-\infty,b]\cap(-\infty,a]^{c}

which gives us {\mathscr{B}_{3}(\mathbb{R})\subseteq\mathscr{B}_{2}(\mathbb{R})}.

For the other direction we need to show {\mathscr{B}(\mathbb{R})\subseteq\mathscr{B}_{3}(\mathbb{R})}. Notice that every open subset of {\mathbb{R}} is a countable union of open intervals and each open interval is a countable union of subintervals of the form {(a,b]} hence {\mathscr{B}(\mathbb{R})\subseteq\mathscr{B}_{3}(\mathbb{R})}.


Now we take a closer look at Borel sets on {\mathbb{R}}. Let’s denote the collection of open subsets by \mathscr{G} and the collection of closed subsets by \mathscr{F}. Then define

\displaystyle G_{\delta}:=\bigcap_{G\in\mathscr{G}} G


\displaystyle F_{\sigma}:=\bigcup_{F\in\mathscr{F}}F

We have the following consequences.


Proposition 4 Every open subset of {\mathbb{R}^{n}} is a {F_{\sigma}} and every closed subset of {\mathbb{R}^{n}} is a {G_{\delta}}.

Proof: Let {F} be a closed subset of {\mathbb{R}^{n}} and define

\displaystyle U_{n}:=\left\{x\in\mathbb{R}^{n}|\left\|x-y\right\|<1/n\;\text{for some }y\in F\right\}

Clearly {F\subseteq \bigcap_{n}U_{n}}. For the other direction, note that by definition each point in {U_{n}} is the limit of sequence of points in {F} and {F} is closed, so such limit should also be in {F}; therefore, {\bigcap_{n}U_{n}\subseteq F}.

Let {U} be an open subset of {\mathbb{R}^{n}} then {U_{c}} is closed. Thus by above {U^{c}} is a {G_{\delta}}, say {U^{c}=\bigcap_{n} U_{n}} where {\left\{U_{n}\right\}} is a sequence of open subsets of {\mathbb{R}^{n}}. Notice {U_{n}^{c}} is open for each {n}, and the identity {U=(\bigcap_{n} U_{n})^{c}=\bigcup_{n} U_{n}^{c}} shows that U is a {F_{\sigma}}.


We end this section by a interesting proposition.


Proposition 5

Definition: Let {(E_{1}, \mathscr{M}_{1})} and {(E_{2}, \mathscr{M}_{2})} be measurable spaces. The product {\sigma}-algebra on {E_{1}\times E_{2}} is defined by

\displaystyle \mathscr{M}_{1}\otimes\mathscr{M}_{2}:=\sigma(\mathscr{M}_{1}\times\mathscr{M}_{2})