Dev Builds » 20230622-0833

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:14:12 1157334 3102 1318 253 1531 +124.33 ± 5.52 0 39 448 1024 40 +275.98 ± 16.11
ncm-dbt-02 10:01:41 1074008 3342 1391 291 1660 +118.78 ± 5.4 1 56 488 1094 32 +263.26 ± 15.43
ncm-dbt-03 10:09:34 1074400 3412 1488 281 1643 +128.46 ± 5.23 0 39 468 1152 47 +288.03 ± 15.76
ncm-dbt-04 10:07:54 1056456 3402 1452 279 1671 +124.91 ± 5.42 0 47 489 1110 55 +273.77 ± 15.42
ncm-dbt-05 10:02:22 1063971 3352 1438 303 1611 +122.48 ± 5.44 2 43 498 1084 49 +268.86 ± 15.27
ncm-dbt-06 10:07:50 1063280 3390 1433 301 1656 +120.64 ± 5.36 0 60 476 1126 33 +268.39 ± 15.62
20000 8520 1708 9772 +123.26 ± 2.2 3 284 2867 6590 256 +272.94 ± 6.36

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
194056 ncm-dbt-02 1234772 58 24 8 26 +98.39 ± 42.75 0 2 9 18 0 +215.71 ± 121.81
194055 ncm-dbt-05 1223598 70 27 6 37 +107.52 ± 40.81 0 2 11 21 1 +225.71 ± 109.18
194054 ncm-dbt-06 1206365 86 39 6 41 +140.5 ± 31.86 0 1 9 32 1 +333.47 ± 126.67
194053 ncm-dbt-01 1200704 88 36 7 45 +118.92 ± 38.93 0 3 11 28 2 +248.32 ± 107.54
194050 ncm-dbt-04 1236917 96 46 9 41 +141.18 ± 29.54 0 0 13 33 2 +322.03 ± 99.97
194049 ncm-dbt-03 1236136 98 41 6 51 +129.79 ± 25.26 0 0 14 35 0 +311.24 ± 95.46
194044 ncm-dbt-02 1228472 500 211 45 244 +119.89 ± 13.51 0 7 74 165 4 +268.17 ± 39.97
194042 ncm-dbt-05 1235072 500 211 55 234 +112.14 ± 14.53 1 5 90 145 9 +236.51 ± 35.94
194039 ncm-dbt-06 1240958 500 219 40 241 +130.14 ± 13.52 0 5 68 170 7 +293.29 ± 41.75
194038 ncm-dbt-01 1228579 500 217 42 241 +126.97 ± 13.94 0 6 71 165 8 +280.42 ± 40.83
194037 ncm-dbt-04 1231206 500 206 42 252 +118.33 ± 13.7 0 7 77 161 5 +261.07 ± 39.14
194036 ncm-dbt-03 1248018 500 222 40 238 +132.54 ± 13.1 0 4 66 174 6 +304.07 ± 42.38
194029 ncm-dbt-02 1235376 500 211 41 248 +123.02 ± 14.82 0 13 60 171 6 +273.0 ± 43.94
194028 ncm-dbt-05 1222678 500 214 44 242 +123.02 ± 14.16 0 9 68 167 6 +273.0 ± 41.7
194027 ncm-dbt-06 1222221 500 210 39 251 +123.81 ± 14.48 0 10 66 167 7 +273.0 ± 42.27
194025 ncm-dbt-04 1217120 500 211 38 251 +125.38 ± 13.61 0 5 74 164 7 +277.93 ± 39.92
194024 ncm-dbt-01 1202858 500 220 33 247 +136.56 ± 13.17 0 5 59 180 6 +318.25 ± 44.96
194023 ncm-dbt-03 1228814 500 216 37 247 +130.14 ± 13.7 0 5 69 168 8 +290.66 ± 41.43
194016 ncm-dbt-02 1236985 500 208 42 250 +119.89 ± 13.68 0 8 72 166 4 +268.17 ± 40.54
194014 ncm-dbt-05 1222114 500 217 49 234 +121.45 ± 13.49 0 6 75 164 5 +270.57 ± 39.67
194013 ncm-dbt-06 1235459 500 206 43 251 +117.55 ± 13.53 0 8 74 165 3 +263.42 ± 39.97
194012 ncm-dbt-03 1233350 500 217 43 240 +126.17 ± 14.46 0 9 66 167 8 +277.93 ± 42.33
194011 ncm-dbt-04 1218902 500 207 42 251 +119.11 ± 14.36 0 8 77 157 8 +256.44 ± 39.15
194010 ncm-dbt-01 1191567 500 204 44 252 +115.22 ± 13.72 0 6 84 154 6 +249.64 ± 37.33
194003 ncm-dbt-02 1232462 500 210 42 248 +121.45 ± 13.49 0 9 66 173 2 +277.93 ± 42.33
194002 ncm-dbt-06 1225805 500 215 40 245 +126.97 ± 13.94 0 9 62 174 5 +288.06 ± 43.65
194001 ncm-dbt-05 1203502 500 220 46 234 +126.17 ± 13.77 0 6 71 166 7 +280.42 ± 40.83
194000 ncm-dbt-04 1231004 500 213 42 245 +123.81 ± 14.48 0 10 66 167 7 +273.0 ± 42.27
193999 ncm-dbt-01 1203314 500 213 30 257 +133.34 ± 13.63 0 6 62 175 7 +304.07 ± 43.81
193998 ncm-dbt-03 1242131 500 212 35 253 +128.55 ± 13.73 0 9 59 178 4 +295.94 ± 44.72
193991 ncm-dbt-02 1229048 500 206 44 250 +116.78 ± 14.37 1 8 74 162 5 +258.75 ± 39.96
193990 ncm-dbt-04 1225820 500 213 32 255 +131.74 ± 14.03 0 8 60 175 7 +298.62 ± 44.44
193989 ncm-dbt-05 1228995 500 215 37 248 +129.35 ± 13.9 0 5 71 165 9 +285.49 ± 40.81
193988 ncm-dbt-06 1211792 500 206 41 253 +119.11 ± 13.69 0 8 73 165 4 +265.78 ± 40.25
193987 ncm-dbt-03 1238212 500 223 45 232 +129.35 ± 12.01 0 2 70 176 2 +304.07 ± 40.91
193986 ncm-dbt-01 1204176 500 212 46 242 +119.89 ± 13.85 0 8 73 164 5 +265.78 ± 40.25
193969 ncm-dbt-02 604854 500 200 42 258 +113.68 ± 13.21 0 4 89 152 5 +247.41 ± 36.02
193968 ncm-dbt-05 645212 500 222 40 238 +132.54 ± 14.36 1 6 61 174 8 +301.33 ± 44.13
193967 ncm-dbt-06 588350 500 217 58 225 +114.45 ± 14.21 0 9 79 156 6 +247.41 ± 38.64
193966 ncm-dbt-04 547196 500 219 47 234 +124.6 ± 14.47 0 5 80 153 12 +263.42 ± 38.27
193964 ncm-dbt-01 789771 500 210 49 241 +116.0 ± 13.37 0 4 87 153 6 +251.89 ± 36.48
193961 ncm-dbt-03 633969 500 218 51 231 +120.67 ± 14.83 0 7 81 150 12 +251.89 ± 38.11
193956 ncm-dbt-01 1237704 14 6 2 6 +102.08 ± 106.14 0 1 1 5 0 +225.71 ± 594.39
193944 ncm-dbt-06 642702 198 78 26 94 +93.43 ± 23.02 0 7 33 59 0 +202.75 ± 60.17
193941 ncm-dbt-05 632521 192 74 20 98 +100.42 ± 23.03 0 3 39 51 3 +205.64 ± 54.88
193938 ncm-dbt-02 672683 196 87 20 89 +123.74 ± 25.31 0 4 29 59 6 +253.28 ± 64.78
193935 ncm-dbt-03 667146 202 89 16 97 +131.49 ± 24.03 0 3 28 64 6 +277.53 ± 66.15
193934 ncm-dbt-04 649751 208 95 20 93 +131.17 ± 24.13 0 4 27 67 6 +277.59 ± 67.25
193932 ncm-dbt-02 842151 20 9 1 10 +147.12 ± 52.02 0 0 2 8 0 +381.52 ± 492.07
193931 ncm-dbt-04 785888 24 10 1 13 +136.91 ± 50.13 0 0 3 9 0 +337.92 ± 423.6
193930 ncm-dbt-05 808355 24 12 2 10 +154.06 ± 67.07 0 0 3 8 1 +337.92 ± 423.6
193929 ncm-dbt-06 825755 28 11 2 15 +115.76 ± 62.86 0 1 3 10 0 +265.1 ± 285.62
193923 ncm-dbt-03 780159 34 15 1 18 +152.04 ± 38.23 0 0 3 14 0 +405.58 ± 467.59
193918 ncm-dbt-05 1217669 66 26 4 36 +120.4 ± 35.64 0 1 9 23 0 +279.59 ± 125.01
193917 ncm-dbt-02 1223285 68 25 6 37 +99.72 ± 35.16 0 1 13 20 0 +219.27 ± 98.79
193916 ncm-dbt-04 1220760 74 32 6 36 +127.48 ± 32.89 0 0 12 24 1 +285.26 ± 103.2
193915 ncm-dbt-03 1236073 78 35 7 36 +130.51 ± 31.66 0 0 12 26 1 +296.12 ± 103.69
193914 ncm-dbt-06 1233402 78 32 6 40 +120.41 ± 35.12 0 2 9 28 0 +279.59 ± 122.83

Commit

Commit ID a49b3ba7ed5d9be9151c8ceb5eed40efe3387c75
Author Linmiao Xu
Date 2023-06-22 08:33:19 UTC
Update default net to nn-5af11540bbfe.nnue Created by retraining the sparsified master net (nn-cd2ff4716c34.nnue) on a 100% minified dataset including Leela transformers data from T80 may2023. Weights permuted with the exact methods and code in: https://github.com/official-stockfish/Stockfish/pull/4620 LEB128 compression done with the new serialize.py param in: https://github.com/glinscott/nnue-pytorch/pull/251 Initially trained with max epoch 800. Around epoch 780, training was paused and max epoch raised to 960. python3 easy_train.py \ --experiment-name L1-1536-sparse-master-retrain \ --training-dataset /data/leela96-dfrc99-v2-T60novdecT77decT78jantosepT79aprmayT80juntonovjan-v6dd-T80febtomay2023.min.binpack \ --early-fen-skipping 27 \ --start-from-engine-test-net True \ --max_epoch 960 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 For preparing the training dataset (interleaved size 328G): python3 interleave_binpacks.py \ leela96-filt-v2.min.binpack \ dfrc99-16tb7p-eval-filt-v2.min.binpack \ filt-v6-dd-min/test60-novdec2021-12tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test77-dec2021-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test78-jantomay2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test78-juntosep2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test79-apr2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test79-may2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jun2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jul2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-aug2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-sep2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-oct2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-nov2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jan2023-16tb7p-filter-v6-dd.min.binpack \ test80-2023/test80-feb2023-16tb7p-no-db.min.binpack \ test80-2023/test80-mar2023-2tb7p-no-db.min.binpack \ test80-2023/test80-apr2023-2tb7p-no-db.min.binpack \ test80-2023/test80-may2023-2tb7p-no-db.min.binpack \ /data/leela96-dfrc99-v2-T60novdecT77decT78jantosepT79aprmayT80juntonovjan-v6dd-T80febtomay2023.min.binpack Minified binpacks and Leela T80 training data from 2023 available at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch879.nnue : 3.9 +/- 5.7 Passed STC: https://tests.stockfishchess.org/tests/view/64928c1bdc7002ce609c7690 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 72000 W: 19242 L: 18889 D: 33869 Ptnml(0-2): 182, 7787, 19716, 8126, 189 Passed LTC: https://tests.stockfishchess.org/tests/view/64930a37dc7002ce609c82e3 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 54552 W: 14978 L: 14647 D: 24927 Ptnml(0-2): 23, 5123, 16650, 5460, 20 closes https://github.com/official-stockfish/Stockfish/pull/4635 bench 2593605
Copyright 2011–2024 Next Chess Move LLC