15.3.1 Markov chains

Problem: The transition matrix of the chain is given, plot the graph with the given probabilities and calculate \( P_{i}(k), i=1,2,3 ; k=1,2 \), assuming that initially the system is in the state of \( S_{1} \). \[ \left(P_{i j}\right)=\left(\begin{array}{cccc} 0.2 & 0.3 & 0.5 & 0 \\ 0.6 & 0 & 0 & 0.4 \\ 0.4 & 0.2 & 0.3 & 0.1 \\ 0.5 & 0 & 0 & 0.5 \end{array}\right) \text {. } \]