For the Markov chain (Xn)n20 with state space S = {1, 2, 3}, transition probability matrix P, and initial state vector p(0) given by 14 5 P = 151723 31 1 7 6 13 25 STER 12 31 5 7 5 13 " 11 p(0) = [23 10 23 = 2 23 " compute the following: (a) the path probability P(Xo = 1, X₁ = 1, X2 = 2, X3 = 3) = (b) three-step transition probability P(X3 = 2 | Xo = 3) = (c) the state vector p(2)

C)

For each of the following transition matrices, do the following: (1) Determine whether the Markov chain is irreducible. (2) Find a stationary distribution 7; is the stationary distribution unique? (3) Give the period of each state. (4) Without using any software package, find P200 approximately. (a) (1/4 1/4 1/2 P = | 1/3 2/3 1/2 1/2, (Ъ) (0.2 0.8 0.5 0.5 P = 0.3 0.7 1

2. Let Zbe a Markov chain on a state space S = {1,… … , m} with transition probability matrix P(i, j) = Tij. Suppose that sample paths of Z are observed from which we record Njj #i →→ j transitions of Z, m N₁ = [N₁; (# transitions out of state i). j=1 It is known that the likelihood function of the observations of Z is given by m Nij ΠΠΠ. i=1_ j=1 L = Incorporating the constraint j = 1 for each i = S, j=1 m (a) Write the loglikelihood function of the observation. (b) Show that the maximum likelihood estimator of i → j transition probability ij is N = Tij = (c) Let m = 2 and assume that the number of transitions of Z is given by the matrix Nij N₁ 50 100 25 75 Use the above information to derive estimators for 11, 12, 721 and 22.