Dev Builds » 20230123-0601

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:01:02 1209079 1674 713 159 802 +119.48 ± 7.83 0 33 237 547 20 +262.24 ± 22.16
ncm-dbt-02 04:57:33 1243705 1650 691 167 792 +114.29 ± 7.81 0 30 260 516 19 +247.48 ± 21.17
ncm-dbt-03 05:00:08 1232000 1668 693 162 813 +114.59 ± 7.64 0 30 258 531 15 +251.16 ± 21.25
ncm-dbt-04 05:00:14 1225144 1676 683 187 806 +105.99 ± 7.75 1 31 293 497 16 +227.03 ± 19.92
ncm-dbt-05 04:58:33 1230400 1662 697 152 813 +118.3 ± 7.43 1 20 257 539 14 +263.53 ± 21.27
ncm-dbt-06 05:00:39 1239070 1670 705 148 817 +120.49 ± 7.53 0 22 254 539 20 +265.26 ± 21.41
ncm-et-3 06:24:19 1301457 1676 681 181 814 +106.9 ± 7.57 0 27 300 495 16 +228.89 ± 19.65
ncm-et-4 06:24:28 1311401 1672 686 164 822 +112.21 ± 7.58 0 31 264 529 12 +246.33 ± 21.01
ncm-et-9 06:25:00 1301484 1658 661 179 818 +104.0 ± 7.85 0 35 294 483 17 +220.33 ± 19.89
ncm-et-10 06:24:47 1293959 1668 680 188 800 +105.62 ± 7.74 0 35 286 499 14 +226.59 ± 20.17
ncm-et-13 06:25:26 1306026 1650 658 172 820 +105.46 ± 7.88 0 36 283 490 16 +224.82 ± 20.28
ncm-et-15 06:24:29 1302119 1676 693 172 811 +111.7 ± 7.7 1 27 278 514 18 +241.6 ± 20.45
20000 8241 2031 9728 +111.56 ± 2.22 3 357 3264 6179 197 +241.69 ± 5.96

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
186284 ncm-dbt-02 1232510 150 63 24 63 +92.45 ± 28.77 0 5 29 38 3 +181.7 ± 64.46
186283 ncm-dbt-05 1221265 162 71 14 77 +127.7 ± 24.94 1 1 20 58 1 +303.87 ± 79.42
186282 ncm-dbt-03 1214543 168 70 14 84 +120.41 ± 24.47 0 4 21 58 1 +272.25 ± 76.45
186281 ncm-dbt-06 1240934 170 72 16 82 +118.88 ± 25.12 0 4 23 56 2 +260.66 ± 73.0
186280 ncm-dbt-01 1194712 174 73 12 89 +127.19 ± 22.01 0 2 23 61 1 +294.38 ± 73.58
186279 ncm-dbt-04 1226659 176 75 18 83 +116.72 ± 25.29 1 3 23 60 1 +268.0 ± 73.0
186278 ncm-dbt-02 1236613 500 207 47 246 +115.22 ± 13.88 0 8 79 158 5 +251.89 ± 38.63
186277 ncm-dbt-05 1231451 500 204 45 251 +114.45 ± 13.21 0 5 85 156 4 +251.89 ± 37.03
186276 ncm-dbt-03 1212135 500 203 48 249 +111.37 ± 14.38 0 12 75 159 4 +243.0 ± 39.61
186275 ncm-dbt-01 1208583 500 209 51 240 +113.68 ± 15.15 0 12 77 152 9 +238.66 ± 39.1
186274 ncm-dbt-06 1251015 500 211 44 245 +120.67 ± 13.67 0 7 74 164 5 +268.17 ± 39.97
186273 ncm-dbt-04 1229229 500 204 58 238 +104.49 ± 13.55 0 7 94 145 4 +223.94 ± 35.14
186272 ncm-dbt-02 1250777 500 216 44 240 +124.6 ± 14.31 0 6 76 158 10 +268.17 ± 39.39
186271 ncm-dbt-05 1228336 500 208 48 244 +115.22 ± 13.38 0 5 85 155 5 +251.89 ± 37.03
186270 ncm-dbt-03 1236843 500 214 48 238 +119.89 ± 12.61 0 2 84 160 4 +268.17 ± 36.99
186269 ncm-dbt-04 1216487 500 206 52 242 +110.6 ± 14.06 0 9 83 153 5 +238.66 ± 37.66
186268 ncm-dbt-06 1234445 500 221 39 240 +132.54 ± 13.28 0 4 67 172 7 +301.33 ± 42.04
186267 ncm-dbt-01 1221529 500 220 45 235 +126.97 ± 14.11 0 9 63 172 6 +285.49 ± 43.31
186266 ncm-dbt-03 1264479 500 206 52 242 +110.6 ± 14.69 0 12 78 154 6 +236.51 ± 38.85
186265 ncm-dbt-05 1240549 500 214 45 241 +122.24 ± 13.83 0 9 67 170 4 +275.45 ± 42.01
186264 ncm-dbt-02 1254921 500 205 52 243 +109.83 ± 13.73 0 11 76 162 1 +245.2 ± 39.38
186263 ncm-dbt-04 1228202 500 198 59 243 +99.2 ± 14.6 0 12 93 139 6 +206.01 ± 35.51
186262 ncm-dbt-01 1211495 500 211 51 238 +115.22 ± 14.05 0 10 74 162 4 +254.16 ± 39.94
186261 ncm-dbt-06 1229888 500 201 49 250 +109.07 ± 13.89 0 7 90 147 6 +232.26 ± 35.99
166334 ncm-et-13 1306373 150 57 13 80 +104.99 ± 24.48 0 2 28 44 1 +226.69 ± 65.45
166333 ncm-et-9 1291575 158 59 15 84 +99.37 ± 25.43 0 3 31 43 2 +205.83 ± 62.08
166332 ncm-et-10 1296760 168 70 24 74 +97.62 ± 25.15 0 4 32 46 2 +202.06 ± 61.25
166331 ncm-et-4 1316377 172 71 9 92 +131.12 ± 21.04 0 1 23 61 1 +307.75 ± 73.44
166330 ncm-et-3 1302345 176 76 21 79 +112.33 ± 21.32 0 2 29 57 0 +254.73 ± 64.6
166329 ncm-et-15 1296382 176 70 25 81 +90.85 ± 25.57 1 5 30 52 0 +201.54 ± 63.3
166328 ncm-et-13 1312551 500 188 43 269 +103.73 ± 14.19 0 12 84 151 3 +223.94 ± 37.43
166327 ncm-et-3 1305617 500 199 61 240 +98.44 ± 13.51 0 9 96 143 2 +211.87 ± 34.83
166326 ncm-et-9 1309083 500 203 55 242 +106.01 ± 14.82 0 12 85 146 7 +221.9 ± 37.21
166325 ncm-et-10 1292828 500 204 65 231 +99.2 ± 13.51 0 11 89 150 0 +217.85 ± 36.33
166324 ncm-et-4 1317452 500 204 46 250 +113.68 ± 14.05 0 9 79 157 5 +247.41 ± 38.64
166323 ncm-et-15 1299205 500 217 46 237 +123.81 ± 14.32 0 7 74 160 9 +268.17 ± 39.97
166322 ncm-et-13 1296821 500 203 56 241 +105.25 ± 13.88 0 9 89 148 4 +226.0 ± 36.29
166321 ncm-et-9 1308668 500 209 63 228 +104.49 ± 14.35 0 11 87 147 5 +221.9 ± 36.76
166320 ncm-et-3 1295013 500 206 53 241 +109.83 ± 14.37 0 8 89 145 8 +230.16 ± 36.26
166319 ncm-et-10 1293309 500 197 53 250 +102.97 ± 14.34 0 11 89 145 5 +217.85 ± 36.33
166318 ncm-et-4 1314564 500 199 50 251 +106.77 ± 14.36 0 13 78 156 3 +232.26 ± 38.82
166317 ncm-et-15 1305039 500 208 50 242 +113.68 ± 13.89 0 8 81 156 5 +247.41 ± 38.13
166316 ncm-et-4 1297211 500 212 59 229 +109.83 ± 13.56 0 8 84 155 3 +240.82 ± 37.4
166315 ncm-et-9 1296612 500 190 46 264 +102.97 ± 13.71 0 9 91 147 3 +221.9 ± 35.86
166314 ncm-et-13 1308361 500 210 60 230 +107.54 ± 15.13 0 13 82 147 8 +223.94 ± 37.87
166313 ncm-et-15 1307852 500 198 51 251 +105.25 ± 13.56 0 7 93 146 4 +226.0 ± 35.35
166312 ncm-et-3 1302853 500 200 46 254 +110.6 ± 14.06 0 8 86 150 6 +236.51 ± 36.93
166311 ncm-et-10 1292942 500 209 46 245 +117.55 ± 14.37 0 9 76 158 7 +254.16 ± 39.42

Commit

Commit ID 596a528c6a9ace6fb1a8407c86d972d96653418d
Author Linmiao Xu
Date 2023-01-23 06:01:32 UTC
Update default net to nn-bc24c101ada0.nnue Created by retraining the master net with Leela T78 data from Aug+Sep 2022 added to the previous best dataset. Trained with end lambda 0.7 and started with max epoch 800. All positions with ply <= 28 were skipped: ``` python easy_train.py \ --experiment-name leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p-sk28-lambda7 \ --training-dataset /data/leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-skip-ply-lteq-28 \ --start-from-engine-test-net True \ --gpus "0," \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 800 ``` Around epoch 750, training was manually paused and max epoch increased to 950 before resuming. The additional Leela training data from T78 was prepared in the same way as the previous best dataset. The exact training data used can be found at: https://robotmoon.com/nnue-training-data/ While the local elo ratings during this experiment were much lower than in recent master nets, several later epochs had a consistent elo above zero, and this was hypothesized to represent potential strength at slower time controls. Local elo at 25k nodes per move leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p-sk28-lambda7 nn-epoch819.nnue : 0.4 +/- 1.1 (nn-bc24c101ada0.nnue) nn-epoch799.nnue : 0.3 +/- 1.2 nn-epoch759.nnue : 0.3 +/- 1.1 nn-epoch839.nnue : 0.2 +/- 1.4 Passed STC https://tests.stockfishchess.org/tests/view/63cabf6f0eefe8694a0c6013 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 41608 W: 11161 L: 10848 D: 19599 Ptnml(0-2): 116, 4496, 11281, 4781, 130 Passed LTC https://tests.stockfishchess.org/tests/view/63cb1856344bb01c191af263 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 76760 W: 20517 L: 20137 D: 36106 Ptnml(0-2): 34, 7435, 23070, 7799, 42 closes https://github.com/official-stockfish/Stockfish/pull/4351 bench 3941848
Copyright 2011–2024 Next Chess Move LLC