Let X₁, be the Markov chain with state space Z and transition probability P2,2+1 = P₁ P2,2-1 = 1- P₁ where p > 1/2. Assume X₁ = 0. (a) Let Y= min{Xo, X₁,...}. What is the distribution of Y? (b) For positive integer k, let T = why e(k) = ke(1). min{n: X₂ = k} and let e(k)= E[T]. Explain (c) Find e(1). Hint: part (b) might be helpful. (d) Use (c) to give another proof that e(1) = ∞o if p = 1/2.

Find the limiting distribution for this Markov chain.

• General notation for Markov chains: P(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p 2 P 1) = 1/², P(X₁ = 1) = 2/₁ 2' 2' P(X₂ = 0) P(X₂ = − 1) = for some 0 < p < 1, p ‡ / . We may denote q=1 - p. That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0. Compute E[S2+1 Sn] for n ≥ 1. (b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1). (c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no, explain why. (d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3). Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem…

C)