This report discusses the application of approximate dynamic programming (ADP) and Krylov subspace methods in solving sequential decision-making problems in machine learning. ADP is used when dealing with large state spaces or when the exact dynamics are unknown. The paper explores various ADP methods such as temporal difference learning and least-squares policy evaluation. The authors also focus on the use of Krylov subspace methods for solving the Bellman equation in ADP. The report provides insights into linear approximation, stochastic algorithms, and the Arnoldi algorithm for orthonormal basis. Overall, the paper highlights the theoretical foundation provided by ADP and its connection with Krylov subspace methods, shedding light on their application in computer science.