Dev Builds » 20221221-0614

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:00:59 1200965 1680 675 190 815 +103.24 ± 7.7 0 38 290 501 11 +222.05 ± 20.04
ncm-dbt-02 04:59:15 1231529 1654 657 176 821 +104.04 ± 8.14 0 41 286 478 22 +217.36 ± 20.18
ncm-dbt-03 05:01:16 1235286 1662 693 159 810 +115.73 ± 7.87 2 25 263 519 22 +251.07 ± 21.04
ncm-dbt-04 05:01:23 1235757 1684 679 181 824 +105.91 ± 7.46 0 28 300 502 12 +228.7 ± 19.65
ncm-dbt-05 04:57:54 1210020 1650 658 176 816 +104.54 ± 7.82 0 34 291 484 16 +222.33 ± 19.99
ncm-dbt-06 05:00:42 1232246 1670 685 210 775 +101.62 ± 7.97 1 41 290 488 15 +215.89 ± 20.04
ncm-et-3 06:24:33 1299890 1674 673 177 824 +106.13 ± 7.98 1 40 274 506 16 +227.39 ± 20.61
ncm-et-4 06:24:20 1302817 1656 676 192 788 +104.59 ± 7.78 1 31 294 487 15 +223.76 ± 19.88
ncm-et-9 06:24:01 1301346 1660 655 166 839 +105.47 ± 7.79 1 33 286 496 14 +226.78 ± 20.17
ncm-et-10 06:24:40 1291908 1668 661 199 808 +98.81 ± 8.16 2 41 304 467 20 +206.19 ± 19.56
ncm-et-13 06:24:49 1298575 1672 671 189 812 +103.08 ± 7.72 0 33 304 483 16 +218.56 ± 19.54
ncm-et-15 06:24:44 1291760 1670 687 161 822 +113.28 ± 7.61 0 26 275 516 18 +245.41 ± 20.56
20000 8070 2176 9754 +105.52 ± 2.26 8 411 3457 5927 197 +225.23 ± 5.79

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
194974 ncm-dbt-05 1210910 150 61 14 75 +112.64 ± 27.44 0 3 25 44 3 +233.67 ± 69.97
194973 ncm-dbt-02 1216844 154 59 13 82 +107.04 ± 25.06 0 1 32 41 3 +219.08 ± 60.01
194972 ncm-dbt-03 1234292 162 67 18 77 +108.48 ± 24.25 0 3 27 50 1 +236.83 ± 67.19
194971 ncm-dbt-06 1226362 170 71 23 76 +100.83 ± 25.02 0 5 28 51 1 +216.32 ± 65.74
194970 ncm-dbt-01 1208110 180 77 22 81 +109.66 ± 24.16 0 3 32 52 3 +229.0 ± 61.3
194969 ncm-dbt-04 1223763 184 76 18 90 +113.37 ± 22.34 0 2 32 56 2 +245.58 ± 61.17
194968 ncm-dbt-05 1208243 500 194 53 253 +100.7 ± 13.85 0 11 89 148 2 +217.85 ± 36.33
194967 ncm-dbt-03 1214054 500 214 50 236 +118.33 ± 13.69 1 6 74 166 3 +268.17 ± 39.97
194966 ncm-dbt-02 1241053 500 203 62 235 +100.7 ± 14.77 0 12 92 139 7 +207.95 ± 35.71
194965 ncm-dbt-01 1199760 500 204 51 245 +109.83 ± 13.05 0 7 84 158 1 +245.2 ± 37.37
194964 ncm-dbt-04 1242874 500 202 53 245 +106.77 ± 13.39 0 7 90 150 3 +232.26 ± 35.99
194963 ncm-dbt-06 1219545 500 201 63 236 +98.44 ± 13.67 0 8 100 138 4 +207.95 ± 33.99
194962 ncm-dbt-05 1223946 500 203 52 245 +108.3 ± 14.21 0 10 84 151 5 +232.26 ± 37.43
194961 ncm-dbt-02 1238025 500 194 47 259 +105.25 ± 14.51 0 12 84 149 5 +223.94 ± 37.43
194960 ncm-dbt-01 1200202 500 204 65 231 +99.2 ± 14.3 0 14 85 149 2 +213.85 ± 37.18
194959 ncm-dbt-03 1250231 500 206 45 249 +116.0 ± 14.21 0 7 83 152 8 +247.41 ± 37.61
194958 ncm-dbt-06 1231177 500 213 61 226 +109.07 ± 14.99 1 12 76 156 5 +236.51 ± 39.32
194957 ncm-dbt-04 1245938 500 198 48 254 +107.54 ± 13.23 0 7 88 153 2 +236.51 ± 36.44
194956 ncm-dbt-05 1196982 500 200 57 243 +102.22 ± 14.33 0 10 93 141 6 +213.85 ± 35.47
194955 ncm-dbt-02 1230197 500 201 54 245 +105.25 ± 15.4 0 16 78 149 7 +219.87 ± 38.7
194954 ncm-dbt-06 1251900 500 200 63 237 +97.69 ± 15.02 0 16 86 143 5 +204.07 ± 36.93
194953 ncm-dbt-01 1195788 500 190 52 258 +98.44 ± 14.74 0 14 89 142 5 +206.01 ± 36.33
194952 ncm-dbt-03 1242568 500 206 46 248 +115.23 ± 15.31 1 9 79 151 10 +243.0 ± 38.64
194951 ncm-dbt-04 1230454 500 203 62 235 +100.7 ± 14.47 0 12 90 143 5 +211.87 ± 36.13
167150 ncm-et-9 1307738 160 65 15 80 +112.33 ± 25.35 1 0 29 48 2 +247.69 ± 63.91
167149 ncm-et-4 1318309 156 68 18 70 +115.42 ± 24.84 0 3 23 51 1 +256.56 ± 73.28
167148 ncm-et-15 1301172 170 69 20 81 +103.06 ± 25.06 0 3 33 46 3 +210.48 ± 60.09
167147 ncm-et-13 1301166 172 73 24 75 +101.79 ± 25.58 0 5 29 50 2 +213.11 ± 64.56
167146 ncm-et-10 1297997 168 66 14 88 +111.18 ± 25.34 0 3 29 49 3 +231.91 ± 64.64
167145 ncm-et-3 1312029 174 63 11 100 +107.09 ± 25.45 0 6 24 56 1 +233.43 ± 70.54
167144 ncm-et-9 1305914 500 197 48 255 +106.77 ± 13.72 0 9 86 152 3 +232.26 ± 36.96
167143 ncm-et-13 1287329 500 204 63 233 +100.7 ± 13.69 0 7 100 138 5 +211.87 ± 33.93
167142 ncm-et-15 1265038 500 204 55 241 +106.77 ± 13.72 0 8 89 149 4 +230.16 ± 36.26
167141 ncm-et-4 1297320 500 196 57 247 +99.2 ± 13.19 0 7 99 142 2 +213.85 ± 34.13
167140 ncm-et-10 1294370 500 193 56 251 +97.69 ± 14.13 0 11 95 140 4 +206.01 ± 35.09
167139 ncm-et-3 1303685 500 207 63 230 +102.97 ± 13.87 0 9 92 145 4 +219.87 ± 35.65
167138 ncm-et-13 1304751 500 198 54 248 +102.97 ± 14.34 0 12 86 148 4 +219.87 ± 36.98
167137 ncm-et-10 1275703 500 197 79 224 +83.57 ± 14.88 0 18 100 128 4 +171.02 ± 34.23
167136 ncm-et-4 1298830 500 209 58 233 +108.3 ± 14.05 0 7 92 144 7 +228.08 ± 35.56
167135 ncm-et-9 1299421 500 193 46 261 +105.25 ± 13.88 0 10 86 151 3 +228.08 ± 36.98
167134 ncm-et-3 1295396 500 198 49 253 +106.77 ± 14.52 0 12 82 151 5 +228.08 ± 37.89
167133 ncm-et-15 1305438 500 202 44 254 +113.68 ± 14.05 0 9 79 157 5 +247.41 ± 38.64
167132 ncm-et-9 1292311 500 200 57 243 +102.22 ± 14.93 0 14 85 145 6 +213.85 ± 37.18
167131 ncm-et-4 1296810 500 203 59 238 +102.97 ± 15.23 1 14 80 150 5 +219.87 ± 38.28
167130 ncm-et-3 1288451 500 205 54 241 +108.3 ± 15.28 1 13 76 154 6 +232.26 ± 39.28
167129 ncm-et-13 1301055 500 196 48 256 +106.01 ± 14.04 0 9 89 147 5 +226.0 ± 36.29
167128 ncm-et-15 1295393 500 212 42 246 +123.02 ± 13.65 0 6 74 164 6 +273.0 ± 39.95
167127 ncm-et-10 1299562 500 205 50 245 +111.37 ± 15.59 2 9 80 150 9 +236.51 ± 38.38

Commit

Commit ID c620886181b5c0fb4a0c067b4173143a81819199
Author Linmiao Xu
Date 2022-12-21 06:14:58 UTC
Update default net to nn-335a9b2d8a80.nnue Created by retraining the master net with a combination of: the previous best dataset (Leela-dfrc_n5000.binpack), with about half the dataset filtered using depth6 multipv2 search to throw away positions where either of the 2 best moves are captures Leela T80 Oct and Nov training data rescored with best moves, adding ~9.5 billion positions Trained effectively the same way as the previous master net: python3 easy_train.py \ --experiment-name=leela-dfrc-filtered-T80-oct-nov \ --training-dataset=/data/leela-dfrc-filtered-T80-oct-nov.binpack \ --start-from-engine-test-net True \ --gpus="0," \ --start-lambda=1.0 \ --end-lambda=0.75 \ --gamma=0.995 \ --lr=4.375e-4 \ --tui=False \ --seed=$RANDOM \ --max_epoch=800 \ --auto-exit-timeout-on-training-finished=900 \ --network-testing-threads 20 \ --num-workers 6 Local testing at a fixed 25k nodes: experiments/experiment_leela-dfrc-filtered-T80-oct-nov/training/run_0/nn-epoch779.nnue localElo: run_0/nn-epoch779.nnue : 4.7 +/- 3.1 The new Leela T80 part of the dataset was prepared by downloading test80 training data from all of Oct 2022 and Nov 2022, rescoring with syzygy 6-piece tablebases and ~600 GB of 7-piece tablebases, saving best moves to exported .plain files, removing all positions with castling flags, then converting to binpacks and using interleave_binpacks.py to merge them together. Scripts used in this data conversion process are available at: https://github.com/linrock/lc0-data-converter Filtering binpack data using depth6 multipv2 search was done by modifying transform.cpp in the tools branch: https://github.com/linrock/Stockfish/tree/tools-filter-multipv2-no-rescore Links for downloading the training data (total size: 338 GB) are available at: https://robotmoon.com/nnue-training-data/ Passed STC: LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 30544 W: 8244 L: 7947 D: 14353 Ptnml(0-2): 93, 3243, 8302, 3542, 92 https://tests.stockfishchess.org/tests/view/63a0d377264a0cf18f86f82b Passed LTC: LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 32464 W: 8866 L: 8573 D: 15025 Ptnml(0-2): 19, 3054, 9794, 3345, 20 https://tests.stockfishchess.org/tests/view/63a10bc9fb452d3c44b1e016 closes https://github.com/official-stockfish/Stockfish/pull/4295 Bench 3554904
Copyright 2011–2024 Next Chess Move LLC