Entropy for clipped actions in PPO is “wrong” in most implementatons? Why not use SAC style squashing?

In policy gradient methods, the actor typically outputs a Gaussian distribution. However, in practice, almost all environments have actions restricted to a certain range.

Almost every implementation of PPO I’ve seen simply clips the action to the allowed range, but uses the unclipped action/distribution when computing log probabilities and entropies. However, this can lead to a failure mode where the distribution means take on high values, making it so the sampled actions are always clipped, killing exploration. The entropy bonus doesn’t do its job because it is computed using the unclipped action, so it stays high even though the actual entropy is very low.

However, this is already pretty much a “solved” issue in implementations of SAC. Implementations of SAC use the tanh function to squash actions to the correct range, and add an adjustment of -log(1 – tanh^2(x)) to the log probabilities to correct for the transformation. They compute entropies using monte-carlo estimation: sampling random actions from the output distribution and taking the mean negative log probability. This is theoretically sound, and very well-established.

So why don’t any implementations of PPO do this? Is the issue of entropy perhaps more of an afterthought in PPO, while it is seen as fundamental to SAC?