Dev Builds » 20230425-0617

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:56:33 583746 4010 1265 791 1954 +41.26 ± 5.09 5 235 1051 709 5 +83.72 ± 10.44
ncm-dbt-02 06:55:12 586724 4000 1294 817 1889 +41.63 ± 5.04 5 229 1051 714 1 +85.23 ± 10.44
ncm-dbt-03 06:57:19 585025 3992 1282 789 1921 +43.13 ± 5.13 3 239 1021 728 5 +87.26 ± 10.62
ncm-dbt-04 06:56:35 568278 4000 1282 741 1977 +47.28 ± 5.11 2 227 1003 764 4 +96.0 ± 10.72
ncm-dbt-05 06:57:19 579932 3998 1277 793 1928 +42.27 ± 5.14 1 244 1036 706 12 +83.8 ± 10.53
20000 6400 3931 9669 +43.11 ± 2.28 16 1174 5162 3621 27 +87.18 ± 4.72

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
427801 ncm-dbt-01 584159 10 1 2 7 -34.8 ± 62.53 0 1 4 0 0 -70.35 ± 134.03 ↓
427800 ncm-dbt-03 585632 492 159 101 232 +41.15 ± 15.15 1 33 119 93 0 +84.98 ± 31.3 ↓
427799 ncm-dbt-05 581943 498 143 86 269 +39.94 ± 14.93 0 35 123 90 1 +79.5 ± 30.77 ↓
427798 ncm-dbt-02 585590 500 150 107 243 +29.95 ± 14.37 1 34 136 79 0 +61.79 ± 29.05 ↓
427797 ncm-dbt-04 569350 500 162 86 252 +53.22 ± 14.15 0 24 127 98 1 +107.54 ± 30.1 ↓
427796 ncm-dbt-01 584874 500 145 103 252 +29.25 ± 14.19 0 36 136 78 0 +58.93 ± 29.06 ↓
427795 ncm-dbt-03 586350 500 159 99 242 +41.89 ± 14.61 0 32 127 90 1 +83.57 ± 30.21 ↓
427794 ncm-dbt-04 569709 500 165 89 246 +53.22 ± 13.59 0 21 132 97 0 +109.07 ± 29.32 ↓
427793 ncm-dbt-02 585337 500 156 98 246 +40.48 ± 13.57 0 26 140 84 0 +82.1 ± 28.36 ↓
427792 ncm-dbt-05 580738 500 165 95 240 +48.96 ± 14.74 0 29 124 95 2 +96.94 ± 30.59 ↓
427791 ncm-dbt-01 583908 500 161 85 254 +53.22 ± 14.69 1 24 125 98 2 +107.54 ± 30.4 ↓
427790 ncm-dbt-03 585632 500 161 89 250 +50.38 ± 14.55 0 28 123 98 1 +101.46 ± 30.72 ↓
427789 ncm-dbt-05 579744 500 164 103 233 +42.6 ± 15.04 0 35 120 94 1 +85.04 ± 31.17 ↓
427788 ncm-dbt-04 567601 500 164 106 230 +40.49 ± 13.99 1 26 137 86 0 +83.57 ± 28.79 ↓
427787 ncm-dbt-02 587197 500 145 111 244 +23.66 ± 14.58 2 35 141 71 1 +48.96 ± 28.41 ↓
427786 ncm-dbt-01 584622 500 170 96 234 +51.8 ± 13.8 0 22 133 94 1 +104.49 ± 29.21 ↓
427785 ncm-dbt-03 585506 500 170 112 218 +40.48 ± 14.39 1 28 134 86 1 +82.1 ± 29.23 ↓
427784 ncm-dbt-02 588387 500 172 89 239 +58.21 ± 13.69 0 20 127 103 0 +119.89 ± 30.01 ↓
427783 ncm-dbt-05 581735 500 172 105 223 +46.84 ± 14.22 0 26 133 89 2 +92.46 ± 29.31 ↓
427782 ncm-dbt-04 569430 500 166 94 240 +50.38 ± 15.58 1 34 107 108 0 +104.49 ± 32.97 ↓
427781 ncm-dbt-01 581693 500 156 102 242 +37.67 ± 15.13 1 36 121 92 0 +77.71 ± 31.04 ↓
427780 ncm-dbt-03 585421 500 154 96 250 +40.48 ± 13.29 0 24 144 82 0 +82.1 ± 27.76 ↓
427779 ncm-dbt-02 586520 500 172 104 224 +47.55 ± 14.13 0 27 128 95 0 +96.94 ± 30.02 ↓
427778 ncm-dbt-05 582277 500 172 94 234 +54.65 ± 13.8 0 22 128 100 0 +112.14 ± 29.92 ↓
427777 ncm-dbt-04 568554 500 155 87 258 +47.55 ± 14.53 0 30 122 98 0 +96.94 ± 30.88 ↓
427776 ncm-dbt-01 583782 500 156 97 247 +41.19 ± 14.43 1 28 133 87 1 +83.57 ± 29.37 ↓
427775 ncm-dbt-03 583196 500 160 103 237 +39.78 ± 14.48 1 28 136 83 2 +79.17 ± 28.96 ↓
427774 ncm-dbt-02 586266 500 169 108 223 +42.6 ± 14.91 0 35 119 96 0 +86.52 ± 31.31 ↓
427773 ncm-dbt-05 578136 500 158 107 235 +35.56 ± 14.74 1 31 137 78 3 +68.99 ± 28.88 ↓
427772 ncm-dbt-04 563314 500 161 88 251 +51.09 ± 14.72 0 29 120 100 1 +102.97 ± 31.15 ↓
427771 ncm-dbt-01 582444 500 156 106 238 +34.86 ± 14.17 2 27 140 81 0 +73.34 ± 28.43 ↓
427770 ncm-dbt-02 589411 500 164 92 244 +50.38 ± 13.86 1 21 133 95 0 +104.49 ± 29.21 ↓
427769 ncm-dbt-03 581527 500 172 102 226 +48.96 ± 14.61 0 30 120 100 0 +99.95 ± 31.16 ↓
427768 ncm-dbt-01 583908 500 163 101 236 +43.3 ± 14.56 0 32 124 94 0 +88.0 ± 30.62 ↓
427767 ncm-dbt-05 575843 500 161 98 241 +44.01 ± 14.33 0 28 133 87 2 +86.52 ± 29.35 ↓
427766 ncm-dbt-04 570388 500 162 105 233 +39.78 ± 14.08 0 28 139 81 2 +77.71 ± 28.54 ↓
427765 ncm-dbt-02 585084 500 166 108 226 +40.49 ± 14.65 1 31 127 91 0 +83.57 ± 30.21 ↓
427764 ncm-dbt-04 567878 500 147 86 267 +42.6 ± 14.91 0 35 119 96 0 +86.52 ± 31.31 ↓
427763 ncm-dbt-01 584327 500 157 99 244 +40.48 ± 14.12 0 29 135 85 1 +80.63 ± 29.1 ↓
427762 ncm-dbt-03 586943 500 147 87 266 +41.89 ± 15.0 0 36 118 96 0 +85.04 ± 31.44 ↓
427761 ncm-dbt-05 579042 500 142 105 253 +25.76 ± 14.34 0 38 138 73 1 +50.38 ± 28.82 ↓

Commit

Commit ID c3ce2204083400267592dc088b8ad9e88aed56b1
Author Linmiao Xu
Date 2023-04-25 06:17:22 UTC
Created by retraining the master net with these changes to the dataset: * Extending v6 filtering to data from T77 dec2021, T79 may2022, and T80 nov2022 * Reducing the number of duplicate positions, prioritizing position scores seen later in time * Using a binpack minimizer to reduce the overall data size Trained the same way as the previous master net, aside from the dataset changes: ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 900 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The new v6-dd filtering reduces duplicate positions by iterating over hourly data files within leela test runs, starting with the most recent, then keeping positions the first time they're seen and ignoring positions that are seen again. This ordering was done with the assumption that position scores seen later in time are generally more accurate than scores seen earlier in the test run. Positions are de-duplicated based on piece orientations, the first token in fen strings. The binpack minimizer was run with default settings after first merging monthly data into single binpacks. ``` python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-filt-v2.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6-dd/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack ``` The code for v6-dd filtering is available along with training data preparation scripts at: https://github.com/linrock/nnue-data Links for downloading the training data components: https://robotmoon.com/nnue-training-data/ The binpack minimizer is from: #4447 Local elo at 25k nodes per move: nn-epoch859.nnue : 1.2 +/- 2.6 Passed STC: https://tests.stockfishchess.org/tests/view/643aad7db08900ff1bc5a832 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 565040 W: 150225 L: 149162 D: 265653 Ptnml(0-2): 1875, 62137, 153229, 63608, 1671 Passed LTC: https://tests.stockfishchess.org/tests/view/643ecf2fa43cf30e719d2042 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 1014840 W: 274645 L: 272456 D: 467739 Ptnml(0-2): 515, 98565, 306970, 100956, 414 closes https://github.com/official-stockfish/Stockfish/pull/4545 bench 3476305
Copyright 2011–2025 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version