Dev Builds » 20230425-0617

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 10:17:02 1008423 3346 1411 294 1641 +120.61 ± 5.47 1 50 499 1077 46 +263.87 ± 15.26
ncm-dbt-02 10:11:07 1037633 3308 1373 322 1613 +114.34 ± 5.44 3 49 529 1040 33 +250.37 ± 14.81
ncm-dbt-03 10:16:40 1034273 3332 1365 297 1670 +115.43 ± 5.4 0 53 528 1049 36 +251.58 ± 14.83
ncm-dbt-04 10:13:50 1032434 3332 1385 312 1635 +116.01 ± 5.44 1 59 504 1070 32 +254.99 ± 15.19
ncm-dbt-05 10:12:04 1034989 3318 1364 288 1666 +116.89 ± 5.44 1 51 516 1053 38 +255.46 ± 15.0
ncm-dbt-06 10:16:43 1020052 3364 1401 325 1638 +115.17 ± 5.34 1 52 530 1068 31 +252.95 ± 14.8
20000 8299 1838 9863 +116.41 ± 2.21 7 314 3106 6357 216 +254.84 ± 6.11

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
185504 ncm-dbt-02 476685 436 176 43 217 +109.47 ± 16.7 3 7 67 136 5 +241.32 ± 41.99
185503 ncm-dbt-05 540936 446 181 43 222 +111.14 ± 15.73 0 10 72 134 7 +234.09 ± 40.49
185502 ncm-dbt-04 496520 458 185 51 222 +104.71 ± 15.57 0 13 74 137 5 +221.55 ± 39.86
185501 ncm-dbt-06 450379 454 176 57 221 +93.24 ± 15.68 0 12 91 117 7 +187.8 ± 35.85
185500 ncm-dbt-03 509368 454 184 45 225 +109.9 ± 14.99 0 8 78 135 6 +233.27 ± 38.84
185499 ncm-dbt-01 501012 456 188 38 230 +118.7 ± 15.11 0 6 75 138 9 +251.0 ± 39.6
185498 ncm-dbt-02 630739 26 10 2 14 +110.44 ± 51.3 0 0 5 8 0 +249.23 ± 175.23
185497 ncm-dbt-05 602580 30 13 0 17 +161.16 ± 37.14 0 0 2 13 0 +458.28 ± 451.16
185496 ncm-dbt-03 574607 30 13 3 14 +120.37 ± 47.07 0 0 5 10 0 +279.52 ± 181.18
185495 ncm-dbt-01 631725 34 15 4 15 +116.54 ± 54.88 0 0 7 9 1 +234.46 ± 137.8
185494 ncm-dbt-06 664015 54 17 6 31 +71.77 ± 38.59 0 1 14 12 0 +150.27 ± 91.33
185493 ncm-dbt-04 669177 54 20 4 30 +106.11 ± 49.34 0 2 8 16 1 +217.63 ± 130.03
185492 ncm-dbt-02 843416 68 32 7 29 +133.99 ± 37.95 0 1 8 24 1 +305.37 ± 135.43
185491 ncm-dbt-05 796819 68 30 8 30 +116.57 ± 35.12 0 0 13 20 1 +250.54 ± 96.9
185489 ncm-dbt-03 780370 68 28 8 32 +105.28 ± 41.62 0 2 11 20 1 +219.27 ± 109.08
185488 ncm-dbt-04 771790 52 22 3 27 +133.09 ± 40.59 0 1 5 20 0 +323.25 ± 188.06
185487 ncm-dbt-01 737968 72 31 9 32 +109.65 ± 37.15 0 1 13 21 1 +231.91 ± 99.22
185486 ncm-dbt-06 753680 56 24 12 20 +75.62 ± 49.52 0 4 8 16 0 +159.18 ± 122.3
185468 ncm-dbt-05 1234167 274 117 32 125 +111.45 ± 18.73 0 4 47 83 3 +240.03 ± 50.26
185467 ncm-dbt-04 1213335 268 117 22 129 +128.74 ± 17.37 0 1 40 90 3 +292.34 ± 54.44
185466 ncm-dbt-02 1231722 278 116 31 131 +109.74 ± 19.33 0 5 48 82 4 +231.6 ± 49.77
185465 ncm-dbt-03 1232851 280 112 26 142 +110.27 ± 19.24 0 7 42 89 2 +240.82 ± 53.29
185464 ncm-dbt-01 1216001 284 123 25 136 +125.02 ± 17.74 0 4 38 98 2 +285.53 ± 56.38
185463 ncm-dbt-06 1210967 300 130 30 140 +120.41 ± 17.96 0 5 43 99 3 +267.37 ± 52.82
185456 ncm-dbt-02 1235329 500 218 46 236 +124.6 ± 13.27 0 7 67 173 3 +285.49 ± 42.08
185455 ncm-dbt-04 1215539 500 211 46 243 +119.11 ± 14.52 0 9 75 158 8 +256.44 ± 39.69
185454 ncm-dbt-05 1222965 500 202 44 254 +113.68 ± 13.72 0 8 80 158 4 +249.64 ± 38.38
185453 ncm-dbt-01 1200520 500 211 51 238 +115.22 ± 13.72 0 7 81 157 5 +251.89 ± 38.11
185452 ncm-dbt-03 1251734 500 197 46 257 +108.3 ± 13.89 0 6 94 143 7 +228.08 ± 35.08
185451 ncm-dbt-06 1230733 500 203 49 248 +110.6 ± 13.73 0 8 84 154 4 +240.82 ± 37.4
185444 ncm-dbt-04 1248919 500 214 47 239 +120.67 ± 13.67 0 7 74 164 5 +268.17 ± 39.97
185443 ncm-dbt-02 1231045 500 199 46 255 +109.83 ± 13.56 0 6 90 149 5 +236.51 ± 35.94
185442 ncm-dbt-03 1263067 500 212 43 245 +122.24 ± 14.33 0 9 70 164 7 +268.17 ± 41.1
185441 ncm-dbt-05 1216555 500 206 37 257 +122.24 ± 14.0 0 8 71 165 6 +270.57 ± 40.83
185440 ncm-dbt-01 1190901 500 214 38 248 +127.76 ± 15.09 1 10 59 172 8 +285.49 ± 44.52
185439 ncm-dbt-06 1216680 500 216 44 240 +124.6 ± 14.31 1 7 67 169 6 +280.42 ± 42.05
185432 ncm-dbt-04 1211987 500 210 49 241 +116.0 ± 13.88 1 8 72 167 2 +263.42 ± 40.52
185431 ncm-dbt-02 1230496 500 214 40 246 +126.17 ± 14.12 0 7 70 165 8 +277.93 ± 41.14
185430 ncm-dbt-03 1222150 500 201 42 257 +114.45 ± 13.38 0 5 86 154 5 +249.64 ± 36.79
185429 ncm-dbt-01 1208593 500 211 45 244 +119.89 ± 13.51 0 6 77 162 5 +265.78 ± 39.12
185428 ncm-dbt-05 1219098 500 199 45 256 +110.6 ± 14.22 0 8 87 148 7 +234.38 ± 36.71
185427 ncm-dbt-06 1235504 500 208 37 255 +123.81 ± 13.1 0 4 76 165 5 +277.93 ± 39.29
185420 ncm-dbt-04 1224458 500 208 40 252 +121.45 ± 14.01 0 8 72 164 6 +268.17 ± 40.54
185419 ncm-dbt-02 1223942 500 202 57 241 +103.73 ± 12.88 0 5 97 146 2 +226.0 ± 34.38
185418 ncm-dbt-05 1233791 500 206 40 254 +119.89 ± 13.33 0 6 76 164 4 +268.17 ± 39.39
185417 ncm-dbt-01 1215108 500 207 35 258 +124.6 ± 14.47 0 8 71 162 9 +270.57 ± 40.83
185416 ncm-dbt-03 1253820 500 210 43 247 +120.67 ± 14.01 0 10 67 169 4 +270.57 ± 41.96
185415 ncm-dbt-06 1216017 500 213 43 244 +123.02 ± 13.29 0 7 69 171 3 +280.42 ± 41.44
185408 ncm-dbt-04 1240186 500 198 50 252 +106.01 ± 13.72 0 10 84 154 2 +232.26 ± 37.43
185407 ncm-dbt-05 1247996 500 210 39 251 +123.81 ± 14.32 1 7 68 168 6 +277.93 ± 41.74
185406 ncm-dbt-02 1235329 500 206 50 244 +112.14 ± 14.38 0 11 77 157 5 +243.0 ± 39.13
185405 ncm-dbt-01 1173985 500 211 49 240 +116.77 ± 14.04 0 8 78 158 6 +254.16 ± 38.89
185404 ncm-dbt-06 1202497 500 214 47 239 +120.67 ± 12.78 0 4 78 165 3 +273.0 ± 38.74
185403 ncm-dbt-03 1220496 500 208 41 251 +120.67 ± 13.32 0 6 75 165 4 +270.57 ± 39.67

Commit

Commit ID c3ce2204083400267592dc088b8ad9e88aed56b1
Author Linmiao Xu
Date 2023-04-25 06:17:22 UTC
Created by retraining the master net with these changes to the dataset: * Extending v6 filtering to data from T77 dec2021, T79 may2022, and T80 nov2022 * Reducing the number of duplicate positions, prioritizing position scores seen later in time * Using a binpack minimizer to reduce the overall data size Trained the same way as the previous master net, aside from the dataset changes: ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 900 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The new v6-dd filtering reduces duplicate positions by iterating over hourly data files within leela test runs, starting with the most recent, then keeping positions the first time they're seen and ignoring positions that are seen again. This ordering was done with the assumption that position scores seen later in time are generally more accurate than scores seen earlier in the test run. Positions are de-duplicated based on piece orientations, the first token in fen strings. The binpack minimizer was run with default settings after first merging monthly data into single binpacks. ``` python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-filt-v2.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6-dd/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack ``` The code for v6-dd filtering is available along with training data preparation scripts at: https://github.com/linrock/nnue-data Links for downloading the training data components: https://robotmoon.com/nnue-training-data/ The binpack minimizer is from: #4447 Local elo at 25k nodes per move: nn-epoch859.nnue : 1.2 +/- 2.6 Passed STC: https://tests.stockfishchess.org/tests/view/643aad7db08900ff1bc5a832 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 565040 W: 150225 L: 149162 D: 265653 Ptnml(0-2): 1875, 62137, 153229, 63608, 1671 Passed LTC: https://tests.stockfishchess.org/tests/view/643ecf2fa43cf30e719d2042 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 1014840 W: 274645 L: 272456 D: 467739 Ptnml(0-2): 515, 98565, 306970, 100956, 414 closes https://github.com/official-stockfish/Stockfish/pull/4545 bench 3476305
Copyright 2011–2024 Next Chess Move LLC