A museum consists of six rooms of equal sizes arranged in the form of a grid with three rows and two columns. Each interior wall has a door that connects to adjacent rooms. Guards move through the rooms through the interior doors. Represent the movements of each guard in the museum as a Markov chain, and show that his states are periodic with period t = 2.

A factory worker will quit with probability 1/2 during her first month, with probability 1/4 during her second month and with probability 1/8 after that. Whenever someone quits, their replacement will start at the beginning of the next month. Model the status of each position as a Markov chain with 3 states. Identify the states and transition matrix. Write down the system of equations determining the long-run proportions. Suppose there are 900 workers in the factory. Find the average number of the workers who have been there for more than 2 months.

Suppose you have two urns with a total of 5 balls. At each step, one of the five balls is chosen at random and switched from its urn to the other urn. Let X, be the number of balls in the first urn after n switches. a) Is {X, : n = 1,2,...} a Markov Chain? Explain. b) Define the state space and provide the one step transition matrix. c) Draw the corresponding transition graph. d) Classify the states. Are there any periodic states? e) What is the probability that, given I have 3 balls in the first urn at the 10th turn, I will have 2 balls at the 12th turn?

Suppose it is known that in the city of Golden the weather is either "good" or "bad". If the weather is good on any given day, there is a 2/3 chance it will be good the next day. If the weather is bad on any given day, there is a 1/2 chance it will be bad the next day. a) Find the stochastic matrix P for this Markov chain. b) Given that on Saturday there is a 100% chance of good weather in Golden, use the stochastic matrix from part (a) to find the probability that the weather on Monday will be good. The initial state xo = c) Over the long run, what is the probability that the weather in Golden is good?