EarlyStopping#
- class delu.tools.EarlyStopping[source]#
Bases:
object
Performs early stopping after N consequtive non-improving metric updates.
A non-improving/unsuccessful metric update is an update with a metric value no better than the best seen metric value over the whole run.
Usage
Preventing overfitting by stopping the training when the validation metric stops improving:
>>> def evaluate_model() -> float: ... # A dummy example. ... return torch.rand(1).item() ... >>> # If the validation score does not increase (mode='max') >>> # for 10 (patience=10) epochs in a row, stop the training. >>> early_stopping = delu.EarlyStopping(patience=10, mode='max') >>> for epoch in range(1000): ... # Training. ... ... ... # Evaluation ... validation_score = evaluate_model() ... ... ... # Submit the new score. ... early_stopping.update(validation_score) ... # Check whether the training should stop. ... if early_stopping.should_stop(): ... break
Additional technical examples:
>>> early_stopping = delu.EarlyStopping(2, mode='max') >>> >>> # Format: (<the best seen score>, <the number of consequtive fails>)``) >>> early_stopping.update(1.0) # (1.0, 0) >>> early_stopping.should_stop() False >>> early_stopping.update(0.0) # (1.0, 1) >>> early_stopping.should_stop() False >>> early_stopping.update(2.0) # (2.0, 0) >>> early_stopping.update(1.0) # (2.0, 1) >>> early_stopping.update(2.0) # (2.0, 2) >>> early_stopping.should_stop() True
Resetting the number of the latest consequtive non-improving updates without resetting the best seen score:
>>> early_stopping.reset_unsuccessful_updates() # (2.0, 0) >>> early_stopping.should_stop() False >>> early_stopping.update(0.0) # (2.0, 1) >>> early_stopping.update(0.0) # (2.0, 2) >>> early_stopping.should_stop() True
The next successfull update resets the number of consequtive fails:
>>> early_stopping.update(0.0) # (2.0, 3) >>> early_stopping.should_stop() True >>> early_stopping.update(3.0) # (3.0, 0) >>> early_stopping.should_stop() False
It is possible to completely reset the instance:
>>> early_stopping.reset() # (-inf, 0) >>> early_stopping.should_stop() False >>> early_stopping.update(-10.0) # (-10.0, 0) >>> early_stopping.update(-100.0) # (-10.0, 1) >>> early_stopping.update(-10.0) # (-10.0, 2) >>> early_stopping.should_stop() True
- __init__( ) None [source]#
- Parameters:
patience – when the number of the latest consequtive non-improving updates reaches
patience
,EarlyStopping.should_stop
starts returningTrue
until the next improving update.mode – if “min”, then the update rule is “the lower value is the better value”. For “max”, it is the opposite.
min_delta – a new value must differ from the current best value by more than
min_delta
to be considered as an improvement.
- reset_unsuccessful_updates() None [source]#
Reset the number of the latest consecutive non-improving updates to zero.
Note that this method does NOT reset the best seen score.
- should_stop() bool [source]#
Check whether the early stopping condition is activated.
See examples in
EarlyStopping
.