Update default net to nn-b1e55edbea57.nnue
Created by retraining the master big net `nn-0000000000a0.nnue` on the same
dataset with the ranger21 optimizer and more WDL skipping at training time.
More WDL skipping is meant to increase lambda accuracy and train on fewer
misevaluated positions where position scores are unlikely to correlate
with game outcomes. Inspired by:
- repeated reports in discord #events-discuss about SF misplaying due to wrong endgame
evals, possibly due to Leela's endgame weaknesses reflected in training data
- an attempt to reduce the skewed dataset piece count distribution where there
are much more positions with less than 16 pieces, since the target piece count
distribution in the trainer is symmetric around 16
The faster convergence seen with ranger21 is meant to:
- prune experiment ideas more quickly since fewer epochs are needed to reach elo maxima
- research faster potential trainings by shortening each run
```yaml
experiment-name: 2560-S7-Re-514G-ranger21-more-wdl-skip
training-dataset: /data/S6-514G.binpack
early-fen-skipping: 28
start-from-engine-test-net: True
nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip
num-epochs: 1200
lr: 4.375e-4
gamma: 0.995
start-lambda: 1.0
end-lambda: 0.7
```
Experiment yaml configs converted to easy_train.sh commands with:
https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py
Implementations based off of Sopel's NNUE training & experimentation log:
https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY
- Experiment 336 - ranger21 https://github.com/Sopel97/nnue-pytorch/tree/experiment_336
- Experiment 351 - more WDL skipping
The version of the ranger21 optimizer used is:
https://github.com/lessw2020/Ranger21/blob/b507df6/ranger21/ranger21.py
The dataset is the exact same as in:
https://github.com/official-stockfish/Stockfish/pull/4782
Local elo at 25k nodes per move:
nn-epoch619.nnue : 6.2 +/- 4.2
Passed STC:
https://tests.stockfishchess.org/tests/view/658a029779aa8af82b94fbe6
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 46528 W: 11985 L: 11650 D: 22893
Ptnml(0-2): 154, 5489, 11688, 5734, 199
Passed LTC:
https://tests.stockfishchess.org/tests/view/658a448979aa8af82b95010f
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 265326 W: 66378 L: 65574 D: 133374
Ptnml(0-2): 153, 30175, 71254, 30877, 204
This was additionally tested with the latest DualNNUE and passed SPRTs:
Passed STC vs. https://github.com/official-stockfish/Stockfish/pull/4919
https://tests.stockfishchess.org/tests/view/658bcd5c79aa8af82b951846
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 296128 W: 76273 L: 75554 D: 144301
Ptnml(0-2): 1223, 35768, 73617, 35979, 1477
Passed LTC vs. https://github.com/official-stockfish/Stockfish/pull/4919
https://tests.stockfishchess.org/tests/view/658c988d79aa8af82b95240f
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 75618 W: 19085 L: 18680 D: 37853
Ptnml(0-2): 45, 8420, 20497, 8779, 68
closes https://github.com/official-stockfish/Stockfish/pull/4942
Bench: 1304666