An imprecise Markov chain is defined by a closed convex set of transition
matrices instead of a unique one for a classical precise Markov chain. These
imprecise Markov chains allow us to model situations where we do not have enough
information to specify a unique transition matrix, or to approximate the
behaviour of non‐stationary Markov chains. We show that there are efficient,
dynamic programming‐ like ways to work and reason with these imprecise Markov
chains; e.g. to calculate the resulting distribution over the states at any time
instant. We prove that this distribution converges in time, similarly to the
precise case and under very mild conditions. We thus effectively prove a
Perron‐Frobenius theorem for a special class of non‐linear systems.