Dev Builds » 20230114-0712

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:59:46 583736 4000 1204 827 1969 +32.84 ± 5.07 1 277 1069 650 3 +65.92 ± 10.36
ncm-dbt-02 07:00:34 586537 4016 1222 810 1984 +35.77 ± 5.19 6 269 1047 679 7 +72.13 ± 10.48
ncm-dbt-03 06:58:38 583962 4000 1188 832 1980 +31.0 ± 5.11 3 284 1071 638 4 +62.33 ± 10.35
ncm-dbt-04 06:59:40 569492 3994 1223 827 1944 +34.56 ± 5.19 2 284 1033 672 6 +69.1 ± 10.57
ncm-dbt-05 06:59:57 578471 3990 1231 800 1959 +37.68 ± 5.25 5 273 1008 704 5 +76.26 ± 10.71
20000 6068 4096 9836 +34.37 ± 2.31 17 1387 5228 3343 25 +69.13 ± 4.69

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
430781 ncm-dbt-02 587494 16 4 4 8 0.0 ± 60.91 0 1 6 1 0 -0.0 ± 125.57 ↓
430780 ncm-dbt-05 575272 490 136 100 254 +25.57 ± 14.84 0 40 130 74 1 +49.98 ± 29.84 ↓
430779 ncm-dbt-04 570989 494 147 97 250 +35.29 ± 14.74 1 32 131 82 1 +71.32 ± 29.66 ↓
430778 ncm-dbt-03 583531 500 161 104 235 +39.78 ± 14.48 0 33 127 90 0 +80.63 ± 30.22 ↓
430777 ncm-dbt-01 584117 500 136 103 261 +22.96 ± 13.59 0 35 147 68 0 +46.13 ± 27.59 ↓
430776 ncm-dbt-02 587028 500 156 96 248 +41.89 ± 14.48 1 27 135 85 2 +83.57 ± 29.08 ↓
430775 ncm-dbt-04 569071 500 139 95 266 +30.65 ± 13.75 0 32 142 76 0 +61.79 ± 28.21 ↓
430774 ncm-dbt-03 586393 500 143 97 260 +32.05 ± 14.12 0 33 139 77 1 +63.23 ± 28.63 ↓
430773 ncm-dbt-05 576701 500 167 95 238 +50.38 ± 14.68 0 30 118 102 0 +102.97 ± 31.44 ↓
430772 ncm-dbt-01 585211 500 152 107 241 +31.35 ± 15.11 0 42 121 87 0 +63.23 ± 31.04 ↓
430771 ncm-dbt-02 586350 500 168 109 223 +41.19 ± 14.95 3 26 131 89 1 +86.52 ± 29.64 ↓
430770 ncm-dbt-03 583614 500 142 99 259 +29.95 ± 13.97 1 30 145 73 1 +60.36 ± 27.79 ↓
430769 ncm-dbt-05 576210 500 158 93 249 +45.42 ± 14.94 1 30 123 95 1 +92.46 ± 30.75 ↓
430768 ncm-dbt-04 569430 500 152 96 252 +39.08 ± 15.21 0 39 116 95 0 +79.17 ± 31.71 ↓
430767 ncm-dbt-01 584201 500 146 89 265 +39.78 ± 14.22 0 31 131 88 0 +80.63 ± 29.67 ↓
430766 ncm-dbt-02 585843 500 153 96 251 +39.78 ± 15.13 0 37 120 92 1 +79.17 ± 31.18 ↓
430765 ncm-dbt-03 583028 500 156 111 233 +31.36 ± 14.34 1 33 136 80 0 +64.66 ± 29.04 ↓
430764 ncm-dbt-04 570709 500 153 102 245 +35.56 ± 14.48 0 34 132 83 1 +70.44 ± 29.57 ↓
430763 ncm-dbt-05 576088 500 156 112 232 +30.65 ± 14.68 1 35 134 79 1 +61.79 ± 29.32 ↓
430762 ncm-dbt-01 582694 500 151 119 230 +22.26 ± 14.98 0 45 129 75 1 +43.3 ± 30.02 ↓
430761 ncm-dbt-02 582527 500 138 95 267 +29.95 ± 14.76 0 40 127 83 0 +60.36 ± 30.26 ↓
430760 ncm-dbt-03 584369 500 145 106 249 +27.15 ± 14.44 0 39 133 78 0 +54.65 ± 29.48 ↓
430759 ncm-dbt-04 569749 500 145 116 239 +20.17 ± 14.83 1 41 138 68 2 +39.08 ± 28.85 ↓
430758 ncm-dbt-05 581402 500 153 101 246 +36.26 ± 13.3 0 26 146 78 0 +73.34 ± 27.55 ↓
430757 ncm-dbt-01 582360 500 161 106 233 +38.37 ± 13.44 0 26 143 81 0 +77.71 ± 27.95 ↓
430756 ncm-dbt-02 586774 500 143 98 259 +31.35 ± 14.07 0 34 137 79 0 +63.23 ± 28.9 ↓
430755 ncm-dbt-04 570589 500 159 102 239 +39.78 ± 15.5 0 40 114 95 1 +79.17 ± 31.97 ↓
430754 ncm-dbt-03 583363 500 154 106 240 +33.46 ± 14.74 0 38 126 86 0 +67.55 ± 30.39 ↓
430753 ncm-dbt-05 577930 500 159 105 236 +37.67 ± 15.5 0 40 118 90 2 +73.34 ± 31.44 ↓
430752 ncm-dbt-01 582861 500 156 103 241 +36.97 ± 14.7 0 36 125 89 0 +74.79 ± 30.51 ↓
430751 ncm-dbt-02 587452 500 151 96 253 +38.37 ± 14.91 1 33 127 88 1 +77.71 ± 30.23 ↓
430750 ncm-dbt-03 584916 500 145 110 245 +24.36 ± 13.84 1 33 146 70 0 +50.38 ± 27.71 ↓
430749 ncm-dbt-04 568236 500 162 117 221 +31.35 ± 14.73 0 38 130 81 1 +61.79 ± 29.86 ↓
430748 ncm-dbt-05 576333 500 145 100 255 +31.36 ± 14.98 1 38 126 85 0 +64.66 ± 30.39 ↓
430747 ncm-dbt-01 584034 500 143 101 256 +29.25 ± 13.79 0 33 142 75 0 +58.93 ± 28.23 ↓
430746 ncm-dbt-02 588387 500 153 110 237 +29.95 ± 14.89 1 36 134 77 2 +58.93 ± 29.33 ↓
430745 ncm-dbt-03 582485 500 142 99 259 +29.95 ± 15.63 0 45 119 84 2 +57.5 ± 31.3 ↓
430744 ncm-dbt-04 567165 500 166 102 232 +44.72 ± 14.1 0 28 130 92 0 +90.97 ± 29.76 ↓
430743 ncm-dbt-05 587834 500 157 94 249 +44.01 ± 15.61 2 34 113 101 0 +92.46 ± 32.12 ↓
430742 ncm-dbt-01 584411 500 159 99 242 +41.89 ± 14.74 1 29 131 87 2 +83.57 ± 29.65 ↓
430741 ncm-dbt-02 586985 500 156 106 238 +34.86 ± 14.44 0 35 130 85 0 +70.44 ± 29.84 ↓

Commit

Commit ID 3d2381d76d7bf9686ef0e0671f60c3b885a7058a
Author Linmiao Xu
Date 2023-01-14 07:12:11 UTC
Update default net to nn-1e7ca356472e.nnue Created by retraining the master net on a dataset composed of: * The Leela-dfrc_n5000.binpack dataset filtered with depth6 multipv2 search to remove positions with only one good move, in addition to removing positions where either of the two best moves are captures * The same Leela T80 oct+nov 2022 training data used in recent best datasets * Additional Leela training data from T60 nov+dec 2021 and T79 apr+may 2022 Trained with end lambda 0.7 and started with max epoch 800. All positions with ply <= 28 were skipped: ``` python easy_train.py \ --experiment-name leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p-sk28-lambda7 \ --training-dataset /data/leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-skip-ply-lteq-28 \ --start-from-engine-test-net True \ --gpus "0," \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 800 ``` Around epoch 780, training was manually paused and max epoch increased to 920 before resuming. During depth6 multipv2 data filtering, positions were considered to have only one good move if the score of the best move was significantly better than the 2nd best move in a way that changes the outcome of the game: * the best move leads to a significant advantage while the 2nd best move equalizes or loses * the best move is about equal while the 2nd best move loses The modified stockfish branch and exact score thresholds used for filtering are at: https://github.com/linrock/Stockfish/tree/tools-filter-multipv2-eval-diff/src/filter About 95% of the Leela portion and 96% of the DFRC portion of the Leela-dfrc_n5000.binpack dataset was filtered. Unfiltered parts of the dataset were left out. The additional Leela training data from T60 nov+dec 2021 and T79 apr+may 2022 was WDL-rescored with about 12TB of syzygy 7-piece tablebases where the material difference is less than around 6 pawns. Best moves were exported to .plain data files during data conversion with the lc0 rescorer. The exact training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move experiment_leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p-sk28-lambda7 run_0/nn-epoch899.nnue : 3.8 +/- 1.6 Passed STC https://tests.stockfishchess.org/tests/view/63bed1f540aa064159b9c89b LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 103344 W: 27392 L: 26991 D: 48961 Ptnml(0-2): 333, 11223, 28099, 11744, 273 Passed LTC https://tests.stockfishchess.org/tests/view/63c010415705810de2deb3ec LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 21712 W: 5891 L: 5619 D: 10202 Ptnml(0-2): 12, 2022, 6511, 2304, 7 closes https://github.com/official-stockfish/Stockfish/pull/4338 bench 4106793
Copyright 2011–2025 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version