D10 · Learning by Trying
A Coach Beside the Player
Say what a critic adds to a policy, and why PPO limits how far each update may move it.
A return of 5 sounds fine. But if this spot usually leads to 8, a 5 is bad news. So compare each return with what you normally get from there. The difference is called the advantage.