Dev Builds » 20240518-0719

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:52:38 584846 4006 1529 608 1869 +81.33 ± 5.06 2 127 837 1022 15 +169.85 ± 11.76
ncm-dbt-02 06:51:19 586572 4000 1480 584 1936 +79.17 ± 5.1 2 135 845 1001 17 +164.29 ± 11.71
ncm-dbt-03 06:51:47 587336 3992 1447 624 1921 +72.67 ± 5.01 1 140 900 945 10 +150.44 ± 11.32
ncm-dbt-04 06:51:20 570871 4002 1508 634 1860 +77.12 ± 5.06 1 138 862 986 14 +159.91 ± 11.58
ncm-dbt-05 06:51:10 585111 4000 1471 619 1910 +75.15 ± 5.0 2 130 895 960 13 +155.76 ± 11.34
20000 7435 3069 9496 +77.09 ± 2.26 8 670 4339 4914 69 +160.0 ± 5.16

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
377224 ncm-dbt-04 569669 2 1 0 1 +189.7 ± 55.98 0 0 0 1 0 +1129.65 ± 376.02
377222 ncm-dbt-01 587707 6 1 1 4 0.0 ± 11.34 0 0 3 0 0 -0.0 ± 10.15
377221 ncm-dbt-03 588302 492 176 73 243 +73.83 ± 14.03 0 17 109 120 0 +155.0 ± 32.67
377220 ncm-dbt-01 585084 500 193 73 234 +85.04 ± 14.2 0 16 99 134 1 +179.9 ± 34.4
377219 ncm-dbt-04 570749 500 191 82 227 +76.97 ± 14.56 0 17 111 118 4 +155.54 ± 32.36
377218 ncm-dbt-02 586054 500 175 71 254 +73.34 ± 14.44 0 21 104 125 0 +153.86 ± 33.54
377217 ncm-dbt-05 580862 500 182 72 246 +77.7 ± 14.86 0 22 97 130 1 +162.35 ± 34.72
377216 ncm-dbt-03 586604 500 177 80 243 +68.27 ± 14.26 0 20 114 115 1 +140.62 ± 31.94
377215 ncm-dbt-04 570749 500 189 81 230 +76.25 ± 15.09 0 24 95 130 1 +158.93 ± 35.03
377214 ncm-dbt-02 583824 500 193 76 231 +82.83 ± 14.72 0 18 100 129 3 +171.02 ± 34.23
377213 ncm-dbt-01 585717 500 186 78 236 +76.25 ± 13.81 0 15 113 121 1 +158.93 ± 31.99
377212 ncm-dbt-05 587240 500 171 74 255 +68.27 ± 14.26 0 19 117 112 2 +138.99 ± 31.46
377211 ncm-dbt-03 588174 500 177 69 254 +76.25 ± 12.89 0 10 122 118 0 +160.64 ± 30.32
377210 ncm-dbt-04 571913 500 191 77 232 +80.63 ± 14.09 1 13 108 127 1 +171.02 ± 32.78
377209 ncm-dbt-05 586646 500 191 90 219 +71.16 ± 13.64 0 14 123 111 2 +145.54 ± 30.38
377208 ncm-dbt-02 588345 500 178 70 252 +76.25 ± 14.67 0 17 113 115 5 +152.18 ± 32.04
377207 ncm-dbt-01 583531 500 196 76 228 +85.04 ± 14.35 1 12 106 128 3 +178.11 ± 33.1
377206 ncm-dbt-03 585970 500 190 71 239 +84.3 ± 13.58 0 11 111 126 2 +176.33 ± 32.16
377205 ncm-dbt-04 569949 500 198 76 226 +86.52 ± 14.95 0 19 93 135 3 +179.9 ± 35.49
377204 ncm-dbt-05 585337 500 178 74 248 +73.34 ± 14.15 1 16 111 122 0 +155.54 ± 32.36
377203 ncm-dbt-01 582569 500 188 65 247 +87.26 ± 14.4 1 14 97 137 1 +187.16 ± 34.76
377202 ncm-dbt-02 584874 500 192 81 227 +78.43 ± 14.31 0 16 110 121 3 +160.64 ± 32.5
377201 ncm-dbt-04 569669 500 181 96 223 +59.64 ± 14.32 0 23 120 106 1 +121.46 ± 31.09
377200 ncm-dbt-03 586816 500 180 81 239 +69.71 ± 15.0 0 24 105 119 2 +142.26 ± 33.37
377199 ncm-dbt-05 582444 500 189 80 231 +76.97 ± 13.83 0 14 115 119 2 +158.93 ± 31.63
377198 ncm-dbt-02 586816 500 190 69 241 +85.78 ± 13.61 0 12 106 131 1 +181.7 ± 33.07
377197 ncm-dbt-01 584201 500 184 67 249 +82.83 ± 13.7 0 14 105 131 0 +176.33 ± 33.3
377196 ncm-dbt-04 572558 500 190 74 236 +82.1 ± 13.37 0 10 116 122 2 +171.02 ± 31.28
377195 ncm-dbt-03 587155 500 172 70 258 +71.88 ± 14.39 0 21 106 123 0 +150.51 ± 33.22
377194 ncm-dbt-05 582485 500 189 71 240 +83.57 ± 14.16 0 16 101 132 1 +176.33 ± 34.04
377193 ncm-dbt-01 582987 500 201 85 214 +82.1 ± 13.98 0 13 111 123 3 +169.27 ± 32.25
377192 ncm-dbt-02 586477 500 178 71 251 +75.52 ± 14.37 1 17 106 126 0 +160.64 ± 33.2
377191 ncm-dbt-04 569789 500 188 79 233 +76.97 ± 14.27 0 17 109 122 2 +158.93 ± 32.69
377190 ncm-dbt-03 588217 500 189 98 213 +63.94 ± 13.94 0 19 122 108 1 +130.94 ± 30.71
377189 ncm-dbt-05 583028 500 193 81 226 +79.17 ± 13.9 0 14 112 122 2 +164.07 ± 32.12
377188 ncm-dbt-01 584327 500 186 82 232 +73.33 ± 15.39 0 25 99 123 3 +148.85 ± 34.34
377187 ncm-dbt-02 586054 500 191 76 233 +81.37 ± 15.1 1 17 102 126 4 +167.53 ± 33.88
377186 ncm-dbt-04 572800 500 179 69 252 +77.71 ± 13.71 0 15 110 125 0 +164.07 ± 32.48
377185 ncm-dbt-03 587452 500 186 82 232 +73.34 ± 14.99 1 18 111 116 4 +148.85 ± 32.4
377184 ncm-dbt-01 587494 500 194 81 225 +79.9 ± 14.64 0 18 104 125 3 +164.07 ± 33.53
377183 ncm-dbt-02 590138 500 183 70 247 +79.9 ± 14.21 0 17 104 128 1 +167.53 ± 33.52
377182 ncm-dbt-05 592850 500 178 77 245 +71.16 ± 14.36 1 15 119 112 3 +145.54 ± 31.07

Commit

Commit ID 1b7dea3f851cd5c5411ba6f07a2f935bfb7da8a9
Author Linmiao Xu
Date 2024-05-18 07:19:10 UTC
Update default main net to nn-c721dfca8cd3.nnue Created by first retraining the spsa-tuned main net `nn-ae6a388e4a1a.nnue` with: - using v6-dd data without bestmove captures removed - addition of T80 mar2024 data - increasing loss by 20% when Q is too high - torch.compile changes for marginal training speed gains And then SPSA tuning weights of epoch 899 following methods described in: https://github.com/official-stockfish/Stockfish/pull/5149 This net was reached at 92k out of 120k steps in this 70+0.7 th 7 SPSA tuning run: https://tests.stockfishchess.org/tests/view/66413b7df9f4e8fc783c9bbb Thanks to @Viren6 for suggesting usage of: - c value 4 for the weights - c value 128 for the biases Scripts for automating applying fishtest spsa params to exporting tuned .nnue are in: https://github.com/linrock/nnue-tools/tree/master/spsa Before spsa tuning, epoch 899 was nn-f85738aefa84.nnue https://tests.stockfishchess.org/tests/view/663e5c893a2f9702074bc167 After initially training with max-epoch 800, training was resumed with max-epoch 1000. ``` experiment-name: 3072--S11--more-data-v6-dd-t80-mar2024--see-ge0-20p-more-loss-high-q-sk28-l8 nnue-pytorch-branch: linrock/nnue-pytorch/3072-r21-skip-more-wdl-see-ge0-20p-more-loss-high-q-torch-compile-more start-from-engine-test-net: False start-from-model: /data/config/apr2024-3072/nn-ae6a388e4a1a.nnue early-fen-skipping: 28 training-dataset: /data/S11-mar2024/: - leela96.v2.min.binpack - test60-2021-11-12-novdec-12tb7p.v6-dd.min.binpack - test78-2022-01-to-05-jantomay-16tb7p.v6-dd.min.binpack - test80-2022-06-jun-16tb7p.v6-dd.min.binpack - test80-2022-08-aug-16tb7p.v6-dd.min.binpack - test80-2022-09-sep-16tb7p.v6-dd.min.binpack - test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - test80-2023-02-feb-16tb7p.v6-sk20.min.binpack - test80-2023-03-mar-2tb7p.v6-sk16.min.binpack - test80-2023-04-apr-2tb7p.v6-sk16.min.binpack - test80-2023-05-may-2tb7p.v6.min.binpack # https://github.com/official-stockfish/Stockfish/pull/4782 - test80-2023-06-jun-2tb7p.binpack - test80-2023-07-jul-2tb7p.binpack # https://github.com/official-stockfish/Stockfish/pull/4972 - test80-2023-08-aug-2tb7p.v6.min.binpack - test80-2023-09-sep-2tb7p.binpack - test80-2023-10-oct-2tb7p.binpack # S9 new data: https://github.com/official-stockfish/Stockfish/pull/5056 - test80-2023-11-nov-2tb7p.binpack - test80-2023-12-dec-2tb7p.binpack # S10 new data: https://github.com/official-stockfish/Stockfish/pull/5149 - test80-2024-01-jan-2tb7p.binpack - test80-2024-02-feb-2tb7p.binpack # S11 new data - test80-2024-03-mar-2tb7p.binpack /data/filt-v6-dd/: - test77-dec2021-16tb7p-filter-v6-dd.binpack - test78-juntosep2022-16tb7p-filter-v6-dd.binpack - test79-apr2022-16tb7p-filter-v6-dd.binpack - test79-may2022-16tb7p-filter-v6-dd.binpack - test80-jul2022-16tb7p-filter-v6-dd.binpack - test80-oct2022-16tb7p-filter-v6-dd.binpack - test80-nov2022-16tb7p-filter-v6-dd.binpack num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 0.8 end-lambda: 0.7 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch899.nnue : 4.6 +/- 1.4 Passed STC: https://tests.stockfishchess.org/tests/view/6645454893ce6da3e93b31ae LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 95232 W: 24598 L: 24194 D: 46440 Ptnml(0-2): 294, 11215, 24180, 11647, 280 Passed LTC: https://tests.stockfishchess.org/tests/view/6645522d93ce6da3e93b31df LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 320544 W: 81432 L: 80524 D: 158588 Ptnml(0-2): 164, 35659, 87696, 36611, 142 closes https://github.com/official-stockfish/Stockfish/pull/5254 bench 1995552
Copyright 2011–2024 Next Chess Move LLC