Note: I wrote these notes for my own reference while I was doing the CS234 Reinforcement Learning course offered by Emma Brunskill at Stanford University in Spring 2024. I hope they are useful for others.