Skip to content
Ahmed Haroon
← Notes

Reinforcement Learning

Policy gradients, value-based methods, model-based control, etc.

  1. 1Imitation LearningBehavioral cloning, distribution shift, dataset aggregation, and multimodal policies.
  2. 2Value-Based Reinforcement LearningFitted Q-iteration, DQN, stability mechanisms, and approximation error.
  3. 3Policy GradientsREINFORCE, variance reduction, policy distributions, and surrogate objectives.
  4. 4Actor-Critic MethodsCritics, multi-step returns, GAE, DDPG, TD3, and soft actor-critic.
  5. 5Advanced Policy OptimizationImportance sampling, trust regions, PPO, natural gradients, and TRPO.
  6. 6Control as InferenceProbabilistic control, variational inference, maximum-entropy RL, and soft actor-critic.
  7. 7Inverse Reinforcement LearningMaximum-entropy IRL, guided cost learning, and adversarial imitation.
  8. 8Reward LearningLearning from preferences, reward identifiability, hacking, and validation.
  9. 9Model-Based Reinforcement LearningLearned and latent dynamics, compounding error, uncertainty, planning, and policy learning.
  10. 10Planning and Monte Carlo Tree SearchAction-sequence optimization, replanning, tree search, and neural guidance.
  11. 11Offline Reinforcement LearningDistribution shift, constrained improvement, IQL, CQL, and expressive policies.
  12. 12ExplorationBandits, optimism, posterior sampling, intrinsic rewards, and curiosity.
  13. 13Skill LearningUnsupervised skill discovery, empowerment, and goal-conditioned behavior.
  14. 14Reinforcement Learning TheoryConvergence, concentration, sample complexity, regret, and function approximation.