Deep Reinforcement Learning

DQN · policy gradients · actor-critic · self-play

1:00

How agents learn from reward alone — DQN, policy gradients, actor-critic, PPO, and the self-play frontier of AlphaGo and AlphaZero.

By the end, you can

Define the MDP framework and explain the Markov property.
Write out the discounted-return formula and explain the role of gamma.
Distinguish value functions from policies, and state when you would prefer each.
Explain why a neural network is needed instead of a lookup table for high-dimensional state spaces.
Describe the DQN architecture, its TD target formula, and how experience replay and the target network each improve stability.
Trace through one DQN training step: act, store, sample, compute target, descend, sync.
Explain the REINFORCE policy-gradient update and why the advantage A = G - V(s) reduces variance.
Compare actor-critic (A2C/A3C) and PPO, including what PPO's clipping accomplishes.
Classify DQN, REINFORCE, A2C, and PPO as on-policy or off-policy and explain the tradeoff.
Identify the three main practical challenges in RL: exploration, sample inefficiency, and reward hacking.
Explain how self-play works and what distinguished AlphaZero from AlphaGo.

Up next in AI, Machine Learning & Course Review