Let X be a square-integrable a random variables. Starting from the projection definition of a conditional expectation, prove that E[X]{Ø, S}] = E[X], E[X]o(X)] = X, and E[g(X)o(X)] = g(X), where g is a Borel function and E[g²(X)] < x.

2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: f(x) = f(x), i=1 where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}. Then prove that E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y = Rd ||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).

Let X be a strictly positive r.v. with the following moment generating function:MX (t) = 1/(1 − 2t)^3(a) Use Markov’s inequality to find an upper bound for P (|X| ≥ 18).(b)  Use Chebyshev’s inequality to find an upper bound for P (|X − 6| ≥ 12).(c)  Find the lowest possible upper bound for P (X ≥ 18) you could get fromChernoff’s inequality.

Assume that S={O1, O2, O3, O4} and that w1 = 0.22, w2 = 0.12, w3 = 0.17, w4 = 0.49. Let E={O2, O3} and F={O3, O4}. (a) What is the probability Pr[E]? (b) What is Pr[F′]? (c) What is Pr[(E∪F)′]?