Notes on Reinforcement Learning

These notes are based on the course of David Silver and are integrated with information from the book of Sutton and Barto and papers available in the literature. The aim is to provide a synthetic overview of the field of reinforcement learning, guiding the reader through its theory. The notes are still at its initial stage and will be possibly improved in the future, for example by summarizing the technical details of recent published works in reinforcement learning. I would appreciate any comment to improve the current version.

The content is organized as follows:

1. Introduction to Reinforcement Learning
2. Modelling the Environment: Markov Process, Markov Reward Process, Markov Decision Process
3. Modelling the Agent with Known Environment: Planning by Dynamic Programming
4. Modelling the Agent with Unknown Environment: Model-Free Prediction
5. Modelling the Agent with Unknown Environment: Model-Free Control
6. Scaling to Large State Spaces: Value Function Approximation
7. Scaling to Large Action Spaces: Policy Gradient
8. Improving the Sample Efficiency of Reinforcement Learning: Model-Based RL
9. Exploration and Exploitation Dilemma