University:
Northwestern University
Course:
IEMS 315 | Stochastic Models And Simulation
Academic year:

2023
Views:

355

Pages:

1
Author:

Leroy George

Homework Assignment 4 The numbers of the problems refer to the 10th edition of Ross. Let P be the transition matrix of a Markov chain. Argue that if for some positive integer r, Pr has all positive entries, then so does Pn for all integers n ≥ r. Hint: Chapman-Kolmogorov equations. (4.31) A certain town never has two sunny days in a row. Each day is classified as being either sunny, cloudy (but dry), or rainy. If it is sunny one day, then it is equally likely to be either cloudy or rainy the next day. If it is rainy or cloudy one day, then there is one chance in two that it will be the same the next day (i.e., w.p. 1/2), and if it changes, then it is equally likely to be either of the other two possibilities. In the long run, what proportion of days are sunny? What proportion are cloudy? (4.42) Let A be a set of states, and the let Ac be the set of the remaining states. (a) What is the interpretation of ∑∑ πi Pi,j i∈A j∈Ac (b) What is the interpretation of ∑∑ πi Pi,j i∈Ac j∈A Explain the identity ∑∑ πi Pi,j = i∈A j∈Ac ∑∑ πi Pi,j i∈Ac j∈A Suppose that on each play of the game a gambler either wins 1 with probability p or loses 1 with probability 1 − p. The gambler continues betting until she or he is either up n or down m. What is the probability that the gambler quits a winner? In the gambler’s ruin problem, suppose the gambler’s fortune is presently i, and suppose that we know that the gambler’s fortune will eventually reach N (before it goes to 0). Given this information, show that the probability he wins the next gamble is p[1−(q/p)i+1 ] , 1−(q/p)i if p = 1/2 i+1 , 2i if p = 1/2 Hint: The probability we want is P(Xn+1 = i + 1|Xn = i, lim Xm = N ) = m→∞ P(Xn+1 = i + 1, limm→∞ Xm = N |Xn = i) P(limm→∞ Xm = N |Xn = i)

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock