Dev Builds » 20230209-0650

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:01:57 1114982 1676 673 137 866 +115.15 ± 7.34 0 22 271 532 13 +254.22 ± 20.7
ncm-dbt-02 04:59:58 1230307 1648 676 173 799 +109.53 ± 7.89 0 35 268 504 17 +235.34 ± 20.85
ncm-dbt-03 05:01:23 1246589 1670 675 142 853 +114.9 ± 7.64 0 24 275 515 21 +248.07 ± 20.55
ncm-dbt-04 05:01:22 1232901 1666 682 161 823 +112.42 ± 7.57 0 24 282 509 18 +242.92 ± 20.28
ncm-dbt-05 04:59:37 1223970 1672 685 152 835 +114.75 ± 7.82 0 35 250 534 17 +250.33 ± 21.58
ncm-dbt-06 05:02:07 1235419 1668 704 165 799 +116.44 ± 7.53 0 24 264 529 17 +255.24 ± 20.99
ncm-et-3 06:26:22 1302657 1668 675 165 828 +109.74 ± 7.83 0 27 295 487 25 +230.97 ± 19.82
ncm-et-4 06:25:29 1304773 1660 657 175 828 +103.87 ± 8.26 0 52 261 500 17 +219.99 ± 21.05
ncm-et-9 06:25:27 1297769 1672 699 187 786 +109.92 ± 8.02 1 31 285 493 26 +231.49 ± 20.2
ncm-et-10 06:26:09 1283405 1662 714 167 781 +118.77 ± 7.73 1 28 242 543 17 +262.82 ± 21.95
ncm-et-13 06:25:53 1310043 1666 690 157 819 +115.2 ± 7.73 1 29 255 532 16 +252.94 ± 21.38
ncm-et-15 06:25:51 1303879 1672 685 163 824 +112.21 ± 7.71 0 27 281 507 21 +240.43 ± 20.34
20000 8215 1944 9841 +112.74 ± 2.24 3 358 3229 6185 225 +243.5 ± 5.99

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
201425 ncm-dbt-02 1229575 148 59 20 69 +93.76 ± 29.1 0 6 25 41 2 +190.85 ± 69.34
201424 ncm-dbt-04 1239524 166 74 11 81 +138.79 ± 23.13 0 2 18 61 2 +326.38 ± 84.25
201423 ncm-dbt-03 1225532 170 68 15 87 +112.04 ± 25.92 0 5 24 54 2 +240.82 ± 71.03
201422 ncm-dbt-06 1215750 168 67 11 90 +120.41 ± 21.75 0 1 27 55 1 +272.25 ± 66.85
201421 ncm-dbt-05 1232115 172 69 14 89 +115.13 ± 23.31 0 3 26 56 1 +256.39 ± 68.68
201420 ncm-dbt-01 1119072 176 72 16 88 +114.52 ± 20.39 0 1 30 57 0 +261.29 ± 63.0
201419 ncm-dbt-02 1241340 500 209 49 242 +115.22 ± 14.69 0 12 72 160 6 +249.64 ± 40.41
201418 ncm-dbt-03 1248426 500 195 36 269 +114.45 ± 13.55 0 6 84 155 5 +249.64 ± 37.33
201417 ncm-dbt-04 1237034 500 212 48 240 +118.33 ± 14.03 0 7 79 157 7 +256.44 ± 38.62
201416 ncm-dbt-06 1234980 500 215 51 234 +118.33 ± 13.52 0 8 73 166 3 +265.78 ± 40.25
201415 ncm-dbt-05 1223411 500 209 43 248 +119.89 ± 14.68 0 10 72 160 8 +258.75 ± 40.49
201414 ncm-dbt-01 1106400 500 207 42 251 +119.11 ± 13.86 0 6 80 157 7 +258.75 ± 38.32
201413 ncm-dbt-03 1270649 500 209 46 245 +117.55 ± 14.04 0 8 77 159 6 +256.44 ± 39.15
201412 ncm-dbt-02 1229705 500 203 62 235 +100.7 ± 13.85 0 8 98 139 5 +211.87 ± 34.38
201411 ncm-dbt-06 1242161 500 214 54 232 +115.22 ± 13.55 0 5 86 153 6 +249.64 ± 36.79
201410 ncm-dbt-05 1220591 500 208 52 240 +112.14 ± 14.38 0 10 80 154 6 +240.82 ± 38.39
201409 ncm-dbt-04 1209460 500 205 58 237 +105.25 ± 13.88 0 8 92 145 5 +223.94 ± 35.61
201408 ncm-dbt-01 1114597 500 199 32 269 +120.67 ± 13.32 0 6 75 165 4 +270.57 ± 39.67
201407 ncm-dbt-03 1241752 500 203 45 252 +113.68 ± 13.89 0 5 90 147 8 +240.82 ± 35.87
201406 ncm-dbt-04 1245586 500 191 44 265 +105.25 ± 13.56 0 7 93 146 4 +226.0 ± 35.35
201405 ncm-dbt-05 1219765 500 199 43 258 +112.14 ± 14.06 0 12 72 164 2 +249.64 ± 40.41
201404 ncm-dbt-02 1220610 500 205 42 253 +117.55 ± 13.87 0 9 73 164 4 +261.07 ± 40.24
201403 ncm-dbt-06 1248788 500 208 49 243 +114.45 ± 14.53 0 10 78 155 7 +245.2 ± 38.89
201402 ncm-dbt-01 1119861 500 195 47 258 +106.01 ± 13.56 0 9 86 153 2 +232.26 ± 36.96
165700 ncm-et-10 1289451 162 66 17 79 +108.48 ± 27.59 0 6 22 51 2 +230.29 ± 73.55
165699 ncm-et-13 1309893 166 75 15 76 +131.51 ± 23.47 0 1 24 55 3 +292.46 ± 71.56
165698 ncm-et-4 1306637 160 60 13 87 +105.15 ± 26.16 0 4 27 47 2 +221.14 ± 67.12
165697 ncm-et-15 1305826 172 74 16 82 +121.92 ± 24.88 0 3 25 55 3 +263.14 ± 70.16
165696 ncm-et-3 1300250 168 68 17 83 +108.9 ± 26.87 0 4 29 47 4 +219.63 ± 64.65
165695 ncm-et-9 1291331 172 70 22 80 +99.59 ± 23.25 0 2 36 46 2 +207.41 ± 56.82
165694 ncm-et-3 1293061 500 193 49 258 +102.97 ± 14.64 0 8 100 132 10 +207.95 ± 33.99
165693 ncm-et-15 1305955 500 205 46 249 +114.45 ± 13.89 0 7 83 154 6 +247.41 ± 37.61
165692 ncm-et-13 1311037 500 207 48 245 +114.45 ± 13.89 0 9 77 160 4 +251.89 ± 39.16
165691 ncm-et-4 1310821 500 200 50 250 +107.54 ± 15.57 0 16 76 150 8 +223.94 ± 39.17
165690 ncm-et-10 1266639 500 226 45 229 +131.74 ± 13.3 0 5 65 174 6 +301.33 ± 42.75
165689 ncm-et-9 1301589 500 205 57 238 +106.01 ± 15.26 0 15 79 149 7 +221.9 ± 38.51
165688 ncm-et-3 1307883 500 214 51 235 +117.55 ± 13.87 0 8 76 161 5 +258.75 ± 39.42
165687 ncm-et-10 1284702 500 207 63 230 +102.97 ± 14.79 1 11 86 147 5 +219.87 ± 36.98
165686 ncm-et-15 1304817 500 202 47 251 +111.37 ± 14.69 0 10 83 149 8 +234.38 ± 37.67
165685 ncm-et-13 1315427 500 207 42 251 +119.11 ± 14.19 0 8 76 159 7 +258.75 ± 39.42
165684 ncm-et-4 1300115 500 201 58 241 +102.22 ± 14.93 0 16 79 151 4 +217.85 ± 38.47
165683 ncm-et-9 1294355 500 207 53 240 +110.6 ± 13.22 0 5 90 151 4 +240.82 ± 35.87
165682 ncm-et-15 1298920 500 204 54 242 +107.54 ± 13.56 0 7 90 149 4 +232.26 ± 35.99
165681 ncm-et-4 1301520 500 196 54 250 +101.46 ± 14.78 0 16 79 152 3 +217.85 ± 38.47
165680 ncm-et-13 1303815 500 201 52 247 +106.78 ± 14.36 1 11 78 158 2 +236.51 ± 38.85
165679 ncm-et-10 1292828 500 215 42 243 +125.38 ± 13.25 0 6 69 171 4 +285.49 ± 41.44
165678 ncm-et-9 1303803 500 217 55 228 +116.78 ± 15.76 1 9 80 147 13 +240.82 ± 38.39
165677 ncm-et-3 1309434 500 200 48 252 +109.07 ± 13.89 0 7 90 147 6 +232.26 ± 35.99

Commit

Commit ID 05dea2ca4657dec10637bb53c4ad583f680e0677
Author Linmiao Xu
Date 2023-02-09 06:50:27 UTC
Update default net to nn-1337b1adec5b.nnue Created by retraining the master net on a dataset composed of: * Most of the previous best dataset filtered to remove positions likely having only one good move * Adding training data from Leela T77 dec2021 rescored with 16tb of 7-piece tablebases Trained with end lambda 0.7 and max epoch 900. Positions with ply <= 28 were removed from most of the previous best dataset before training began. A new nnue-pytorch trainer param for skipping early plies was used to skip plies <= 24 in the unfiltered and additional Leela T77 parts of the dataset. ``` python easy_train.py \ --experiment-name leela96-dfrc99-T80octnovT79aprmayT60novdec-eval-filt-v2-T78augsep-12tb-T77dec-16tb-lambda7-sk24 \ --training-dataset /data/leela96-dfrc99-T80octnovT79aprmayT60novdec-eval-filt-v2-T78augsep-12tb-T77dec-16tb.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/easy-train-early-fen-skipping \ --early-fen-skipping 24 \ --gpus "0," \ --start-from-engine-test-net True \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 900 ``` The depth6 multipv2 search filtering method is the same as the one used for filtering recent best datasets, with a lower eval difference threshold to remove slightly more positions than before. These parts of the dataset were filtered: * 96% of T60T70wIsRightFarseerT60T74T75T76.binpack * 99% of dfrc_n5000.binpack * T80 oct + nov 2022 data, no positions with castling flags, rescored with ~600gb 7p tablebases * T79 apr + may 2022 data, rescored with 12tb 7p tablebases * T60 nov + dec 2021 data, rescored with 12tb 7p tablebases These parts of the dataset were not filtered. Positions with ply <= 24 were skipped during training: * T78 aug + sep 2022 data, rescored with 12tb 7p tablebases * 84% of T77 dec 2021 data, rescored with 16tb 7p tablebases The code and exact evaluation thresholds used for data filtering can be found at: https://github.com/linrock/Stockfish/tree/tools-filter-multipv2-eval-diff-t2/src/filter The exact training data used can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch859.nnue : 3.5 +/ 1.2 Passed STC: LLR: 2.95 (-2.94,2.94) <0.00,2.00> https://tests.stockfishchess.org/tests/view/63dfeefc73223e7f52ad769f Total: 219744 W: 58572 L: 58002 D: 103170 Ptnml(0-2): 609, 24446, 59284, 24832, 701 Passed LTC: https://tests.stockfishchess.org/tests/view/63e268fc73223e7f52ade7b6 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 91256 W: 24528 L: 24121 D: 42607 Ptnml(0-2): 48, 8863, 27390, 9288, 39 closes https://github.com/official-stockfish/Stockfish/pull/4387 bench 3841998
Copyright 2011–2024 Next Chess Move LLC