Chapter 9.BMO, Problem 1BMO

The transition matrix for a Markov process is

$T=\begin{array}{c}\text{State}\text{1}\\ \text{State}2\end{array}\stackrel{\begin{array}{cc}S\text{tate}1& S\text{tate}2\end{array}}{\left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]}$

and the initial-state distribution vector is

$X_0=\begin{array}{c}\text{State}\text{1}\\ \text{State}\text{2}\end{array}\left[\begin{array}{c}.6\\ .4\end{array}\right]$

Find $X_2$.

To determine

To find:

The vector $X_2$.

Answer to Problem 1BMO

Solution:

The vector $X_2$ is $\left[\begin{array}{c}0.366\\ 0.634\end{array}\right]$

Explanation of Solution

Given:

The transition matrix for a Markov process is

$T=\begin{array}{c}\text{State}\text{1}\\ \text{State}2\end{array}\stackrel{\begin{array}{cc}S\text{tate}1& S\text{tate}2\end{array}}{\left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]}$

and the initial-state distribution vector is

$X_0=\begin{array}{c}\text{State}\text{1}\\ \text{State}\text{2}\end{array}\left[\begin{array}{c}.6\\ .4\end{array}\right]$

Approach:

From the given data,

Write the expression of the probability distribution after one observation

$X_1=T{X}_0\to \left(1\right)$

Write the expression of the probability distribution after two observations

$X_2=T{X}_1\to \left(2\right)$

Calculation:

Substitute $\left[\begin{array}{c}.6\\ .4\end{array}\right]$ for $X_0$ and $\left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]$ for $T$ in equation (1)

$\begin{array}{rll}{X}_1& =& \left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]\cdot \left[\begin{array}{c}.6\\ .4\end{array}\right]\\ & =& \left[\begin{array}{c}\left(.3\times .6\right)+\left(.4\times .4\right)\\ \left(.7\times .6\right)+\left(.6\times .4\right)\end{array}\right]\\ & =& \left[\begin{array}{c}.18+.16\\ .42+.24\end{array}\right]\\ & =& \left[\begin{array}{c}.34\\ .66\end{array}\right]\end{array}$

Similarly, Substitute $\left[\begin{array}{c}.34\\ .66\end{array}\right]$ for $X_1$ and $\left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]$ for $T$ in equation (2).

$\begin{array}{rll}{X}_2& =& \left[\begin{array}{cc}.3& .4\\ .7& .6\end{array}\right]\cdot \left[\begin{array}{c}.34\\ .66\end{array}\right]\\ & =& \left[\begin{array}{c}\left(.3\times .34\right)+\left(.4\times .66\right)\\ \left(.7\times .34\right)+\left(.6\times .66\right)\end{array}\right]\\ & =& \left[\begin{array}{c}.102+.264\\ .238+.396\end{array}\right]\\ & =& \left[\begin{array}{c}.366\\ .634\end{array}\right]\end{array}$

Conclusion:

Hence, vector $X_2$ is $\left[\begin{array}{c}.366\\ .634\end{array}\right]$.