Dev Builds » 20240107-2015

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:50:58 584578 4000 1428 628 1944 +70.44 ± 4.97 0 145 918 929 8 +145.54 ± 11.2
ncm-dbt-02 06:52:58 585992 4016 1433 635 1948 +69.97 ± 5.01 1 143 937 911 16 +143.05 ± 11.07
ncm-dbt-03 06:50:55 587044 4000 1469 628 1903 +74.15 ± 4.84 1 111 945 932 11 +153.65 ± 10.97
ncm-dbt-04 06:52:51 570455 3996 1408 636 1952 +67.98 ± 4.85 0 125 990 869 14 +138.74 ± 10.69
ncm-dbt-05 06:53:18 584779 3988 1420 644 1924 +68.48 ± 5.08 3 150 923 904 14 +140.48 ± 11.17
20000 7158 3171 9671 +70.2 ± 2.21 5 674 4713 4545 63 +144.26 ± 4.93

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
402377 ncm-dbt-02 585295 16 7 4 5 +65.88 ± 61.23 0 0 5 3 0 +136.9 ± 147.45
402376 ncm-dbt-05 585211 488 174 80 234 +67.77 ± 14.19 0 18 115 110 1 +139.46 ± 31.71
402375 ncm-dbt-04 572114 496 183 75 238 +76.88 ± 13.75 0 12 119 114 3 +156.98 ± 30.89
402374 ncm-dbt-03 587240 500 184 72 244 +79.17 ± 13.14 0 10 119 120 1 +165.8 ± 30.8
402373 ncm-dbt-01 582152 500 163 88 249 +52.51 ± 14.79 0 27 124 96 3 +102.97 ± 30.57
402372 ncm-dbt-02 586139 500 178 96 226 +57.5 ± 14.36 0 24 121 104 1 +116.78 ± 30.96
402371 ncm-dbt-05 582694 500 176 71 253 +74.06 ± 14.03 0 18 109 123 0 +155.54 ± 32.7
402370 ncm-dbt-03 586689 500 184 77 239 +75.52 ± 13.64 0 14 116 119 1 +157.24 ± 31.47
402369 ncm-dbt-04 569230 500 180 85 235 +66.82 ± 13.18 0 13 130 106 1 +137.37 ± 29.29
402368 ncm-dbt-01 584159 500 169 79 252 +63.23 ± 14.06 0 21 118 111 0 +130.94 ± 31.35
402367 ncm-dbt-02 583992 500 181 68 251 +79.9 ± 14.78 0 17 108 120 5 +160.64 ± 32.85
402366 ncm-dbt-05 581652 500 174 78 248 +67.55 ± 13.94 1 12 131 102 4 +135.76 ± 29.14
402365 ncm-dbt-04 570629 500 179 83 238 +67.54 ± 14.23 0 19 118 111 2 +137.37 ± 31.31
402364 ncm-dbt-03 587622 500 188 77 235 +78.43 ± 14.17 0 16 109 123 2 +162.35 ± 32.67
402363 ncm-dbt-02 587240 500 176 68 256 +76.25 ± 13.96 0 15 114 119 2 +157.24 ± 31.83
402362 ncm-dbt-01 585928 500 179 76 245 +72.61 ± 14.27 0 18 113 117 2 +148.85 ± 32.06
402361 ncm-dbt-03 587622 500 191 78 231 +79.9 ± 13.16 0 9 121 118 2 +165.8 ± 30.41
402360 ncm-dbt-05 581070 500 189 96 215 +65.38 ± 15.1 1 20 119 105 5 +129.35 ± 31.2
402359 ncm-dbt-04 569989 500 170 76 254 +66.1 ± 13.0 0 11 136 101 2 +134.15 ± 28.29
402358 ncm-dbt-01 584327 500 182 82 236 +70.44 ± 14.48 0 22 106 122 0 +147.19 ± 33.22
402357 ncm-dbt-02 584159 500 174 75 251 +69.71 ± 14.03 0 17 119 112 2 +142.26 ± 31.1
402356 ncm-dbt-05 587028 500 181 63 256 +83.57 ± 14.16 0 16 101 132 1 +176.33 ± 34.04
402355 ncm-dbt-04 568832 500 165 76 259 +62.51 ± 13.15 0 15 131 104 0 +129.35 ± 29.24
402354 ncm-dbt-03 587579 500 182 80 238 +71.88 ± 14.25 0 18 114 116 2 +147.19 ± 31.9
402353 ncm-dbt-02 587240 500 175 84 241 +63.95 ± 13.94 1 16 125 107 1 +132.54 ± 30.2
402352 ncm-dbt-01 582235 500 187 79 234 +76.25 ± 13.05 0 10 123 116 1 +158.93 ± 30.17
402351 ncm-dbt-03 587197 500 181 78 241 +72.61 ± 12.93 0 11 125 114 0 +152.18 ± 29.92
402350 ncm-dbt-05 586393 500 190 84 226 +74.79 ± 14.49 0 19 108 121 2 +153.86 ± 32.88
402349 ncm-dbt-02 586858 500 180 77 243 +72.61 ± 14.41 0 19 111 118 2 +148.85 ± 32.4
402348 ncm-dbt-04 568355 500 170 83 247 +61.07 ± 14.52 0 23 119 106 2 +123.02 ± 31.23
402347 ncm-dbt-01 585253 500 197 84 219 +79.9 ± 13.92 0 14 111 123 2 +165.8 ± 32.28
402346 ncm-dbt-04 569549 500 174 76 250 +68.99 ± 14.0 0 17 120 111 2 +140.62 ± 30.95
402345 ncm-dbt-03 585632 500 185 89 226 +67.55 ± 14.37 1 18 116 114 1 +140.62 ± 31.61
402344 ncm-dbt-01 585759 500 175 70 255 +74.06 ± 14.03 0 18 109 123 0 +155.54 ± 32.7
402343 ncm-dbt-02 584537 500 179 69 252 +77.7 ± 13.86 0 15 111 123 1 +162.35 ± 32.31
402342 ncm-dbt-05 586223 500 170 87 243 +58.21 ± 14.25 0 24 119 107 0 +119.89 ± 31.25
402034 ncm-dbt-01 586816 500 176 70 254 +74.79 ± 13.62 0 15 114 121 0 +157.24 ± 31.83
402033 ncm-dbt-05 587962 500 166 85 249 +56.78 ± 14.6 1 23 121 104 1 +116.78 ± 30.96
402032 ncm-dbt-03 586774 500 174 77 249 +68.27 ± 13.68 0 15 125 108 2 +138.99 ± 30.12
402031 ncm-dbt-04 574947 500 187 82 231 +74.06 ± 13.89 0 15 117 116 2 +152.18 ± 31.35
402030 ncm-dbt-02 588472 500 183 94 223 +62.51 ± 14.17 0 20 123 105 2 +126.18 ± 30.59

Commit

Commit ID 584d9efedcde330eeb96a99215552ddfb06f52ba
Author Linmiao Xu
Date 2024-01-07 20:15:52 UTC
Dual NNUE with L1-128 smallnet Credit goes to @mstembera for: - writing the code enabling dual NNUE: https://github.com/official-stockfish/Stockfish/pull/4898 - the idea of trying L1-128 trained exclusively on high simple eval positions The L1-128 smallnet is: - epoch 399 of a single-stage training from scratch - trained only on positions from filtered data with high material difference - defined by abs(simple_eval) > 1000 ```yaml experiment-name: 128--S1-only-hse-v2 training-dataset: - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-1k.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack # T80 2022 - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-1k.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.binpack # T80 2023 - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-1k.binpack start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: https://github.com/official-stockfish/nnue-pytorch/pull/259 Data filtered for high simple eval positions with: https://github.com/linrock/nnue-data/blob/32d6a68/filter_high_simple_eval_plain.py https://github.com/linrock/Stockfish/blob/61dbfe/src/tools/transform.cpp#L626-L655 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch399.nnue : -318.1 +/- 2.1 Passed STC: https://tests.stockfishchess.org/tests/view/6574cb9d95ea6ba1fcd49e3b LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 62432 W: 15875 L: 15521 D: 31036 Ptnml(0-2): 177, 7331, 15872, 7633, 203 Passed LTC: https://tests.stockfishchess.org/tests/view/6575da2d4d789acf40aaac6e LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64830 W: 16118 L: 15738 D: 32974 Ptnml(0-2): 43, 7129, 17697, 7497, 49 closes https://github.com/official-stockfish/Stockfish/pulls Bench: 1330050 Co-Authored-By: mstembera <5421953+mstembera@users.noreply.github.com>
Copyright 2011–2025 Next Chess Move LLC