I am new to Markov Processes, and while learning the discrete Markov chain with the following matrix $ $ P=\begin{bmatrix}\frac{1}{3} & \frac{2}{3} & 0 & 0 \ \frac{1}{2} & \frac{1}{2} & 0 & 0 \ \frac{1}{4} & 0 & \frac{1}{4} & \frac{1}{2} \0 & 0 & 0 & 1\end{bmatrix},$ $

I was told that $ f_{34}(n)=\left(\frac14\right)^{n-1}\frac12$ , where $ f_{ij}(n)$ denotes the probability that starting in state $ i$ , we visit $ j$ for the first time at time $ n$ . I can also verify this by hand.

I’ve tried the following input in Mathematica:

`p = {{1/3, 2/3, 0, 0}, {1/2, 1/2, 0, 0}, {1/4, 0, 1/4, 1/2}, {0, 0, 0, 1}}; P = DiscreteMarkovProcess[3, p]; f34 = FirstPassageTimeDistribution[P, 4]; PDF[f34, n] `

But the result is $ $ \left\{\begin{array}{ll} 3 \times 4^{-n} & n>0 \0 & \text { True }\end{array}\right.$ $

I’m confused since $ f_{34}(1)=p_{34}=\frac12$ , how can it be $ \frac34$ as given? I’m using the version 12.1.1.0, could anyone give me a hint where I went wrong? Thanks!