Dev Builds » 20240402-0649

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:51:36 584692 4000 1471 626 1903 +74.52 ± 4.98 3 132 888 971 6 +155.97 ± 11.4
ncm-dbt-02 06:51:21 586099 4000 1441 596 1963 +74.52 ± 5.12 3 156 838 999 4 +156.39 ± 11.77
ncm-dbt-03 06:52:53 586785 4010 1477 603 1930 +76.96 ± 4.95 2 123 889 981 10 +160.6 ± 11.38
ncm-dbt-04 06:53:16 571442 4000 1423 602 1975 +72.34 ± 5.05 3 139 906 938 14 +149.26 ± 11.28
ncm-dbt-05 06:53:25 585662 3990 1412 666 1912 +65.73 ± 5.06 0 164 932 888 11 +134.32 ± 11.12
20000 7224 3093 9683 +72.81 ± 2.25 11 714 4453 4777 45 +151.22 ± 5.09

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
377395 ncm-dbt-03 585337 10 5 1 4 +147.05 ± 74.68 0 0 1 4 0 +381.35 ± 516.06 ↓
377394 ncm-dbt-05 586350 490 170 86 234 +60.15 ± 15.24 0 28 106 110 1 +122.54 ± 33.2 ↓
377393 ncm-dbt-04 571351 500 179 69 252 +77.71 ± 14.72 2 14 108 124 2 +164.07 ± 32.83 ↓
377392 ncm-dbt-01 585632 500 181 78 241 +72.61 ± 13.24 0 13 121 116 0 +152.18 ± 30.64 ↓
377391 ncm-dbt-02 586308 500 175 69 256 +74.79 ± 14.06 0 17 111 121 1 +155.54 ± 32.36 ↓
377390 ncm-dbt-03 587664 500 182 76 242 +74.79 ± 14.2 1 15 112 121 1 +157.24 ± 32.18 ↓
377389 ncm-dbt-05 588047 500 183 96 221 +61.07 ± 14.92 0 26 113 109 2 +123.02 ± 32.15 ↓
377388 ncm-dbt-01 586097 500 186 77 237 +76.98 ± 14.13 1 15 108 126 0 +164.07 ± 32.83 ↓
377387 ncm-dbt-04 572840 500 183 73 244 +77.7 ± 14.99 0 20 104 122 4 +157.24 ± 33.54 ↓
377386 ncm-dbt-03 587834 500 184 69 247 +81.37 ± 13.2 0 11 113 126 0 +172.78 ± 31.83 ↓
377385 ncm-dbt-02 588856 500 189 91 220 +68.99 ± 14.42 1 18 114 116 1 +143.89 ± 31.92 ↓
377384 ncm-dbt-01 583237 500 181 75 244 +74.79 ± 14.34 0 18 110 120 2 +153.86 ± 32.54 ↓
377383 ncm-dbt-04 569509 500 184 75 241 +76.98 ± 14.13 1 14 111 123 1 +162.35 ± 32.31 ↓
377382 ncm-dbt-05 581777 500 176 78 246 +68.99 ± 14.0 0 17 120 111 2 +140.62 ± 30.95 ↓
377381 ncm-dbt-02 585379 500 194 67 239 +90.22 ± 14.31 0 16 92 141 1 +192.71 ± 35.72 ↓
377380 ncm-dbt-03 586520 500 187 77 236 +77.71 ± 14.72 1 17 105 125 2 +162.35 ± 33.37 ↓
377379 ncm-dbt-01 585801 500 181 91 228 +63.23 ± 14.06 0 21 118 111 0 +130.94 ± 31.35 ↓
377378 ncm-dbt-02 580530 500 179 75 246 +73.33 ± 14.15 0 18 111 120 1 +152.18 ± 32.38 ↓
377377 ncm-dbt-04 568156 500 185 76 239 +76.97 ± 14.56 0 18 108 121 3 +157.24 ± 32.87 ↓
377376 ncm-dbt-05 585970 500 176 74 250 +71.88 ± 14.39 0 19 112 117 2 +147.19 ± 32.24 ↓
377375 ncm-dbt-03 586901 500 186 73 241 +79.9 ± 15.06 0 20 101 125 4 +162.35 ± 34.05 ↓
377374 ncm-dbt-01 583070 500 177 78 245 +69.71 ± 14.31 1 18 112 119 0 +147.19 ± 32.24 ↓
377373 ncm-dbt-04 570469 500 175 72 253 +72.61 ± 13.99 0 17 114 118 1 +150.51 ± 31.88 ↓
377372 ncm-dbt-05 583405 500 189 94 217 +66.82 ± 13.92 0 17 123 108 2 +135.76 ± 30.5 ↓
377371 ncm-dbt-02 587877 500 174 68 258 +74.79 ± 14.34 0 20 104 126 0 +157.24 ± 33.54 ↓
377370 ncm-dbt-03 587240 500 179 78 243 +71.16 ± 13.93 0 17 116 116 1 +147.19 ± 31.57 ↓
377369 ncm-dbt-01 583950 500 188 75 237 +79.9 ± 13.92 0 15 108 126 1 +167.53 ± 32.81 ↓
377368 ncm-dbt-05 583950 500 171 88 241 +58.21 ± 13.4 0 18 131 101 0 +119.89 ± 29.37 ↓
377367 ncm-dbt-04 571752 500 165 80 255 +59.64 ± 14.18 0 22 122 105 1 +121.46 ± 30.78 ↓
377366 ncm-dbt-03 585759 500 186 77 237 +76.97 ± 13.68 0 13 117 118 2 +158.93 ± 31.27 ↓
377365 ncm-dbt-02 586604 500 174 72 254 +71.88 ± 14.53 0 22 104 124 0 +150.51 ± 33.54 ↓
377364 ncm-dbt-01 584117 500 188 78 234 +77.7 ± 14.72 0 21 99 129 1 +162.35 ± 34.39 ↓
377363 ncm-dbt-02 583656 500 182 79 239 +72.61 ± 14.69 1 20 104 125 0 +153.86 ± 33.54 ↓
377362 ncm-dbt-05 589197 500 173 63 264 +77.7 ± 14.29 0 18 105 126 1 +162.35 ± 33.37 ↓
377361 ncm-dbt-04 572679 500 175 86 239 +62.51 ± 14.17 0 20 123 105 2 +126.18 ± 30.59 ↓
377360 ncm-dbt-03 585295 500 189 74 237 +81.37 ± 13.66 0 14 107 129 0 +172.78 ± 32.96 ↓
377359 ncm-dbt-01 585632 500 189 74 237 +81.37 ± 13.96 1 11 112 124 2 +171.02 ± 32.04 ↓
377358 ncm-dbt-02 589582 500 174 75 251 +69.71 ± 15.26 1 25 98 126 0 +147.19 ± 34.48 ↓
377357 ncm-dbt-03 588515 500 179 78 243 +71.16 ± 13.64 0 16 117 117 0 +148.85 ± 31.38 ↓
377356 ncm-dbt-05 586604 500 174 87 239 +61.07 ± 14.1 0 21 122 106 1 +124.6 ± 30.76 ↓
377355 ncm-dbt-04 574784 500 177 71 252 +74.79 ± 13.47 0 14 116 120 0 +157.24 ± 31.47 ↓

Commit

Commit ID 0716b845fdef8a20102b07eaec074b8da8162523
Author Viren6
Date 2024-04-02 06:49:48 UTC
Update NNUE architecture to SFNNv9 and net nn-ae6a388e4a1a.nnue Part 1: PyTorch Training, linrock Trained with a 10-stage sequence from scratch, starting in May 2023: https://github.com/linrock/nnue-tools/blob/master/exp-sequences/3072-10stage-SFNNv9.yml While the training methods were similar to the L1-2560 training sequence, the last two stages introduced min-v2 binpacks, where bestmove capture and in-check position scores were not zeroed during minimization, for compatibility with skipping SEE >= 0 positions and future research. Training data can be found at: https://robotmoon.com/nnue-training-data This net was tested at epoch 679 of the 10th training stage: https://tests.stockfishchess.org/tests/view/65f32e460ec64f0526c48dbc Part 2: SPSA Training, Viren6 The net was then SPSA tuned. This consisted of the output weights (32 * 8) and biases (8) as well as the L3 biases (32 * 8) and L2 biases (16 * 8), totalling 648 params in total. The SPSA tune can be found here: https://tests.stockfishchess.org/tests/view/65fc33ba0ec64f0526c512e3 With the help of Disservin , the initial weights were extracted with: https://github.com/Viren6/Stockfish/tree/new228 The net was saved with the tuned weights using: https://github.com/Viren6/Stockfish/tree/new241 Earlier nets of the SPSA failed STC compared to the base 3072 net of part 1: https://tests.stockfishchess.org/tests/view/65ff356e0ec64f0526c53c98 Therefore it is suspected that the SPSA at VVLTC has added extra scaling on top of the scaling of increasing the L1 size. Passed VVLTC 1: https://tests.stockfishchess.org/tests/view/6604a9020ec64f0526c583da LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 53042 W: 13554 L: 13256 D: 26232 Ptnml(0-2): 12, 5147, 15903, 5449, 10 Passed VVLTC 2: https://tests.stockfishchess.org/tests/view/660ad1b60ec64f0526c5dd23 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 17506 W: 4574 L: 4315 D: 8617 Ptnml(0-2): 1, 1567, 5362, 1818, 5 STC Elo estimate: https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0 Elo: -7.66 ± 3.8 (95%) LOS: 0.0% Total: 9618 W: 2440 L: 2652 D: 4526 Ptnml(0-2): 80, 1281, 2261, 1145, 42 nElo: -13.94 ± 6.9 (95%) PairsRatio: 0.87 closes https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0 bench 1823302 Co-Authored-By: Linmiao Xu <lin@robotmoon.com>
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version