TensorFLow: Gradient Clipping

Posted 2021-02-22 yaos

The parameters clipnorm and clipvalue can be used with all optimizers to control gradient clipping。
Keras的所有optimizer都可以使用clipnorm和clipvalue来防止梯度过大。

from keras import optimizers

# All parameter gradients will be clipped to
# a maximum norm of 1.
sgd = optimizers.SGD(lr=0.01, clipnorm=1.)

from keras import optimizers

# All parameter gradients will be clipped to
# a maximum value of 0.5 and
# a minimum value of -0.5.
sgd = optimizers.SGD(lr=0.01, clipvalue=0.5)