Dev Builds » 20230101-1128

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:00:47 1102726 1664 681 181 802 +107.72 ± 7.71 1 30 283 504 14 +232.97 ± 20.27
ncm-dbt-02 04:57:30 1196042 1650 649 170 831 +103.85 ± 7.85 0 36 289 485 15 +221.1 ± 20.07
ncm-dbt-03 05:01:03 1236991 1672 685 162 825 +112.45 ± 7.44 0 22 285 513 16 +244.35 ± 20.15
ncm-dbt-04 05:01:09 1235419 1680 675 175 830 +106.63 ± 7.81 0 35 288 499 18 +226.94 ± 20.1
ncm-dbt-05 04:58:52 1235055 1658 656 162 840 +106.76 ± 7.4 0 24 298 496 11 +231.54 ± 19.69
ncm-dbt-06 05:01:11 1229958 1676 711 172 793 +115.85 ± 7.45 2 19 269 534 14 +256.95 ± 20.77
ncm-et-3 06:25:27 1303314 1668 676 173 819 +108.13 ± 7.67 0 29 290 498 17 +231.6 ± 20.01
ncm-et-4 06:25:46 1299275 1664 671 169 824 +108.18 ± 7.81 0 30 291 490 21 +229.18 ± 19.98
ncm-et-9 06:26:43 1304514 1670 688 179 803 +109.37 ± 7.61 0 30 280 511 14 +236.95 ± 20.38
ncm-et-10 06:26:03 1293806 1672 681 183 808 +106.72 ± 8.01 0 39 281 495 21 +225.27 ± 20.36
ncm-et-13 06:25:38 1309442 1658 657 181 820 +102.63 ± 7.58 0 29 308 479 13 +219.11 ± 19.38
ncm-et-15 06:25:32 1297245 1668 654 212 802 +94.32 ± 7.88 5 32 323 464 10 +202.14 ± 18.94
20000 8084 2119 9797 +106.87 ± 2.22 8 355 3485 5968 184 +229.59 ± 5.76

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
201761 ncm-dbt-02 1197069 150 69 16 65 +128.28 ± 22.98 0 1 21 52 1 +296.83 ± 77.05
201760 ncm-dbt-05 1217320 158 59 10 89 +111.41 ± 23.71 0 3 24 52 0 +252.04 ± 71.63
201759 ncm-dbt-01 1107741 164 68 15 81 +116.45 ± 23.11 0 1 29 50 2 +253.0 ± 64.01
201758 ncm-dbt-03 1237141 172 76 20 76 +117.38 ± 24.93 0 3 27 53 3 +249.8 ± 67.28
201757 ncm-dbt-04 1255878 180 64 18 98 +90.8 ± 23.82 0 5 35 49 1 +190.85 ± 58.47
201756 ncm-dbt-06 1225033 176 74 23 79 +103.64 ± 22.96 0 4 29 55 0 +229.93 ± 64.7
201755 ncm-dbt-02 1195970 500 208 51 241 +112.91 ± 14.38 0 9 82 152 7 +240.82 ± 37.9
201754 ncm-dbt-05 1240046 500 184 49 267 +96.19 ± 12.65 0 5 106 138 1 +207.95 ± 32.65
201753 ncm-dbt-01 1101525 500 206 51 243 +111.37 ± 13.89 0 10 78 159 3 +245.2 ± 38.89
201752 ncm-dbt-04 1213193 500 211 56 233 +111.37 ± 14.06 0 8 85 151 6 +238.66 ± 37.17
201751 ncm-dbt-03 1232130 500 203 53 244 +107.54 ± 13.56 0 7 90 149 4 +232.26 ± 35.99
201750 ncm-dbt-06 1238139 500 212 49 239 +117.55 ± 13.36 0 6 79 161 4 +261.07 ± 38.58
201749 ncm-dbt-02 1198197 500 184 49 267 +96.19 ± 13.48 0 8 102 137 3 +204.07 ± 33.61
201748 ncm-dbt-05 1242451 500 208 51 241 +112.91 ± 14.22 0 8 84 151 7 +240.82 ± 37.4
201747 ncm-dbt-01 1088947 500 205 57 238 +106.01 ± 14.67 1 9 87 147 6 +226.0 ± 36.75
201746 ncm-dbt-04 1239706 500 194 55 251 +99.2 ± 14.6 0 12 93 139 6 +206.01 ± 35.51
201745 ncm-dbt-03 1236027 500 203 45 252 +113.68 ± 13.72 0 7 83 155 5 +247.41 ± 37.61
201744 ncm-dbt-06 1247544 500 210 51 239 +114.45 ± 13.72 1 5 82 158 4 +254.16 ± 37.82
201743 ncm-dbt-02 1192932 500 188 54 258 +95.44 ± 15.13 0 18 84 144 4 +200.24 ± 37.29
201742 ncm-dbt-05 1240405 500 205 52 243 +109.83 ± 13.56 0 8 84 155 3 +240.82 ± 37.4
201741 ncm-dbt-01 1112693 500 202 58 240 +102.97 ± 13.87 0 10 89 148 3 +221.9 ± 36.31
201740 ncm-dbt-04 1232899 500 206 46 248 +115.22 ± 14.21 0 10 75 160 5 +251.89 ± 39.67
201739 ncm-dbt-03 1242667 500 203 44 253 +114.45 ± 13.21 0 5 85 156 4 +251.89 ± 37.03
201738 ncm-dbt-06 1209117 500 215 49 236 +119.89 ± 13.85 1 4 79 160 6 +265.78 ± 38.54
166886 ncm-et-13 1312794 158 65 18 75 +106.57 ± 23.71 0 3 26 50 0 +238.09 ± 68.56
166885 ncm-et-4 1298227 164 59 14 91 +97.84 ± 23.07 0 3 31 48 0 +214.24 ± 62.19
166884 ncm-et-3 1301784 168 71 20 77 +108.9 ± 23.69 0 3 28 52 1 +238.25 ± 65.91
166883 ncm-et-15 1298010 168 60 26 82 +71.3 ± 31.22 3 6 31 42 2 +154.14 ± 61.78
166882 ncm-et-9 1307712 170 64 18 88 +96.41 ± 26.39 0 8 23 54 0 +210.48 ± 70.82
166881 ncm-et-10 1293846 172 80 18 74 +131.12 ± 22.94 0 1 25 57 3 +292.0 ± 69.98
166880 ncm-et-13 1312653 500 189 50 261 +99.2 ± 13.99 0 10 95 141 4 +209.91 ± 35.06
166879 ncm-et-15 1293658 500 202 65 233 +97.69 ± 13.34 1 6 99 143 1 +213.85 ± 34.13
166878 ncm-et-3 1301881 500 208 50 242 +113.68 ± 13.72 0 8 80 158 4 +249.64 ± 38.38
166877 ncm-et-4 1291262 500 203 53 244 +107.54 ± 15.13 0 10 91 138 11 +217.85 ± 35.89
166876 ncm-et-10 1290855 500 209 57 234 +109.07 ± 13.73 0 8 86 152 4 +236.51 ± 36.93
166875 ncm-et-9 1299757 500 211 57 232 +110.6 ± 14.06 0 8 86 150 6 +236.51 ± 36.93
166874 ncm-et-13 1298979 500 200 55 245 +103.73 ± 13.05 0 6 95 147 2 +226.0 ± 34.87
166873 ncm-et-4 1314373 500 201 49 250 +109.07 ± 13.89 0 9 84 153 4 +236.51 ± 37.42
166872 ncm-et-15 1293070 500 199 64 237 +96.19 ± 14.71 1 11 95 138 5 +202.15 ± 35.11
166871 ncm-et-9 1305131 500 202 52 246 +107.54 ± 13.4 0 7 89 151 3 +234.38 ± 36.22
166870 ncm-et-3 1309240 500 193 50 257 +102.22 ± 14.02 0 9 94 142 5 +215.85 ± 35.23
166869 ncm-et-10 1292136 500 200 56 244 +102.97 ± 15.52 0 16 82 144 8 +211.87 ± 37.79
166868 ncm-et-3 1300352 500 204 53 243 +108.3 ± 14.37 0 9 88 146 7 +228.08 ± 36.51
166867 ncm-et-13 1313345 500 203 58 239 +103.73 ± 14.5 0 10 92 141 7 +215.85 ± 35.68
166866 ncm-et-4 1293241 500 208 53 239 +111.37 ± 14.06 0 8 85 151 6 +238.66 ± 37.17
166865 ncm-et-15 1304243 500 193 57 250 +96.94 ± 13.49 0 9 98 141 2 +207.95 ± 34.43
166864 ncm-et-9 1305456 500 211 52 237 +114.45 ± 13.72 0 7 82 156 5 +249.64 ± 37.86
166863 ncm-et-10 1298389 500 192 52 256 +99.95 ± 14.91 0 14 88 142 6 +207.95 ± 36.54

Commit

Commit ID be9bc420afa11314c886c0aede4c4ae3d76f8a50
Author Linmiao Xu
Date 2023-01-01 11:28:51 UTC
Update default net to nn-60fa44e376d9.nnue Created by retraining the master net on the previous best dataset with additional filtering. No new data was added. More of the Leela-dfrc_n5000.binpack part of the dataset was pre-filtered with depth6 multipv2 search to remove bestmove captures. About 93% of the previous Leela/SF data and 99% of the SF dfrc data was filtered. Unfiltered parts of the dataset were left out. The new Leela T80 oct+nov data is the same as before. All early game positions with ply count <= 28 were skipped during training by modifying the training data loader in nnue-pytorch. Trained in a similar way as recent master nets, with a different nnue-pytorch branch for early ply skipping: python3 easy_train.py \ --experiment-name=leela93-dfrc99-filt-only-T80-oct-nov-skip28 \ --training-dataset=/data/leela93-dfrc99-filt-only-T80-oct-nov.binpack \ --start-from-engine-test-net True \ --nnue-pytorch-branch=linrock/nnue-pytorch/misc-fixes-skip-ply-lteq-28 \ --gpus="0," \ --start-lambda=1.0 \ --end-lambda=0.75 \ --gamma=0.995 \ --lr=4.375e-4 \ --tui=False \ --seed=$RANDOM \ --max_epoch=800 \ --network-testing-threads 20 \ --num-workers 6 For the exact training data used: https://robotmoon.com/nnue-training-data/ Details about the previous best dataset: #4295 Local testing at a fixed 25k nodes: experiment_leela93-dfrc99-filt-only-T80-oct-nov-skip28 Local Elo: run_0/nn-epoch779.nnue : 5.1 +/- 1.5 Passed STC https://tests.stockfishchess.org/tests/view/63adb3acae97a464904fd4e8 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 36504 W: 9847 L: 9538 D: 17119 Ptnml(0-2): 108, 3981, 9784, 4252, 127 Passed LTC https://tests.stockfishchess.org/tests/view/63ae0ae25bd1e5f27f13d884 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 36592 W: 10017 L: 9717 D: 16858 Ptnml(0-2): 17, 3461, 11037, 3767, 14 closes https://github.com/official-stockfish/Stockfish/pull/4314 bench 4015511
Copyright 2011–2024 Next Chess Move LLC