Reinforcement Learning for Execution¶
Under development
This optional advanced module is part of the course scaffold.
This module examines reinforcement learning applied to order execution — the one area of trading where RL has a credible production track record, because the problem is genuinely sequential, the action space is well-defined, and dense reward signals exist. It is equally a module about where RL does not work in trading and why. It is for learners who completed the RL material in Part VII and want to see the honest gap between the papers and practice.
Topics¶
- Formulating execution as an MDP: state (inventory, time, book features), actions (child-order placement), reward (shortfall)
- Why execution is RL-friendly while strategy discovery mostly is not: signal density, stationarity, and episode structure
- Value-based and policy-gradient methods applied to the execution problem
- Simulator fidelity: market replay versus reactive simulators, and the sim-to-real gap for impact
- Benchmarking rigorously against TWAP and Almgren–Chriss baselines — the bar an RL policy must clear
- Evaluation pitfalls: overfitting to replayed history, reward hacking, and non-stationarity across regimes