1 Table of Contents: “...Discrete Markov Processes and Numerical Algorithms for Markov Chains -- Markov Decision Processes...”