Dev Builds » 20230922-1726

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:57:12 1196890 3336 1408 299 1629 +120.06 ± 5.46 1 53 491 1082 41 +263.94 ± 15.39
ncm-dbt-02 09:51:17 1199745 3314 1408 283 1623 +122.82 ± 5.34 0 41 493 1080 43 +271.17 ± 15.35
ncm-dbt-03 09:57:31 1238197 3358 1437 295 1626 +123.06 ± 5.42 0 51 481 1101 46 +271.04 ± 15.55
ncm-dbt-04 09:59:06 1224726 3344 1458 272 1614 +128.82 ± 5.36 1 41 449 1133 48 +288.85 ± 16.1
ncm-dbt-05 09:51:00 1225082 3288 1438 296 1554 +125.91 ± 5.22 0 35 468 1105 36 +283.42 ± 15.75
ncm-dbt-06 09:59:42 1223654 3360 1436 329 1595 +118.9 ± 5.36 1 43 527 1066 43 +259.87 ± 14.83
20000 8585 1774 9641 +123.24 ± 2.19 3 264 2909 6567 257 +272.82 ± 6.31

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
203948 ncm-dbt-05 1217823 288 118 23 147 +119.05 ± 17.28 0 2 48 91 3 +262.76 ± 49.48
203947 ncm-dbt-02 1194708 314 129 21 164 +124.58 ± 15.9 0 4 41 112 0 +293.22 ± 54.22
203946 ncm-dbt-01 1213565 336 152 20 164 +144.24 ± 17.02 0 4 35 122 7 +333.36 ± 58.89
203945 ncm-dbt-04 1214566 344 157 26 161 +139.32 ± 15.24 0 3 38 128 3 +333.47 ± 56.53
203944 ncm-dbt-06 1239239 360 151 31 178 +120.41 ± 17.57 0 6 56 110 8 +253.15 ± 46.07
203943 ncm-dbt-03 1236865 358 155 37 166 +118.96 ± 16.3 0 3 61 109 6 +255.13 ± 43.78
203942 ncm-dbt-05 1216638 500 224 44 232 +130.94 ± 13.51 0 4 70 168 8 +293.29 ± 41.07
203941 ncm-dbt-02 1193428 500 208 52 240 +112.14 ± 14.53 0 10 81 152 7 +238.66 ± 38.15
203940 ncm-dbt-01 1177227 500 209 48 243 +116.0 ± 14.69 1 10 71 163 5 +256.44 ± 40.73
203939 ncm-dbt-04 1217002 500 219 49 232 +123.02 ± 14.33 0 6 78 156 10 +263.42 ± 38.85
203938 ncm-dbt-06 1217571 500 217 53 230 +118.33 ± 13.52 0 6 79 160 5 +261.07 ± 38.58
203937 ncm-dbt-03 1252839 500 218 40 242 +129.35 ± 13.9 0 8 62 174 6 +293.29 ± 43.72
203936 ncm-dbt-05 1228111 500 215 40 245 +126.97 ± 12.47 0 4 69 175 2 +295.94 ± 41.39
203935 ncm-dbt-02 1194856 500 210 39 251 +123.81 ± 14.32 0 7 74 160 9 +268.17 ± 39.97
203934 ncm-dbt-01 1212405 500 201 50 249 +108.3 ± 13.56 0 8 86 153 3 +236.51 ± 36.93
203933 ncm-dbt-06 1245655 500 222 54 224 +121.45 ± 12.58 0 3 79 165 3 +275.45 ± 38.39
203932 ncm-dbt-04 1199177 500 212 42 246 +123.02 ± 13.29 0 6 72 168 4 +277.93 ± 40.53
203931 ncm-dbt-03 1211825 500 216 44 240 +124.6 ± 15.12 0 9 72 157 12 +263.42 ± 40.52
203930 ncm-dbt-05 1225023 500 218 38 244 +130.94 ± 12.95 0 5 64 177 4 +304.07 ± 43.1
203929 ncm-dbt-02 1195136 500 217 38 245 +130.14 ± 13.88 0 5 70 166 9 +288.06 ± 41.11
203928 ncm-dbt-01 1184320 500 220 36 244 +134.15 ± 13.97 0 4 69 166 11 +295.94 ± 41.39
203927 ncm-dbt-03 1258191 500 210 45 245 +119.11 ± 13.86 0 9 71 166 4 +265.78 ± 40.81
203926 ncm-dbt-04 1244396 500 222 28 250 +142.25 ± 14.66 0 7 55 175 13 +318.25 ± 46.48
203925 ncm-dbt-06 1212821 500 211 43 246 +121.45 ± 13.31 0 4 80 160 6 +268.17 ± 38.21
203924 ncm-dbt-05 1236459 500 227 49 224 +129.35 ± 13.54 0 5 69 169 7 +290.66 ± 41.43
203923 ncm-dbt-02 1196331 500 209 38 253 +123.81 ± 13.1 0 3 79 162 6 +275.45 ± 38.39
203922 ncm-dbt-04 1230439 500 226 45 229 +131.74 ± 13.49 0 5 66 172 7 +298.62 ± 42.41
203921 ncm-dbt-01 1198071 500 207 47 246 +115.22 ± 14.05 0 9 77 159 5 +251.89 ± 39.16
203920 ncm-dbt-03 1241579 500 214 35 251 +130.14 ± 13.52 0 5 68 170 7 +293.29 ± 41.75
203919 ncm-dbt-06 1205034 500 211 44 245 +120.67 ± 13.67 0 8 71 167 4 +270.57 ± 40.83
203918 ncm-dbt-05 1233063 500 218 52 230 +119.89 ± 13.68 0 7 75 163 5 +265.78 ± 39.69
203917 ncm-dbt-02 1190317 500 221 46 233 +126.97 ± 13.76 0 6 70 167 7 +282.94 ± 41.13
203916 ncm-dbt-01 1197347 500 207 47 246 +115.22 ± 14.05 0 8 80 156 6 +249.64 ± 38.38
203915 ncm-dbt-04 1230995 500 213 39 248 +126.18 ± 14.62 1 7 67 167 8 +280.42 ± 42.05
203914 ncm-dbt-03 1238391 500 206 41 253 +119.11 ± 14.03 0 9 72 164 5 +263.42 ± 40.52
203913 ncm-dbt-06 1234102 500 218 51 231 +120.67 ± 14.51 1 5 79 156 9 +261.07 ± 38.58
203912 ncm-dbt-05 1218460 500 218 50 232 +121.45 ± 14.17 0 8 73 162 7 +265.78 ± 40.25
203911 ncm-dbt-02 1233441 500 214 49 237 +119.11 ± 13.52 0 6 78 161 5 +263.42 ± 38.85
203910 ncm-dbt-01 1195301 500 212 51 237 +116.0 ± 14.05 0 10 73 163 4 +256.44 ± 40.21
203909 ncm-dbt-03 1227691 500 218 53 229 +119.11 ± 14.03 0 8 75 161 6 +261.07 ± 39.69
203908 ncm-dbt-04 1236512 500 209 43 248 +119.89 ± 13.33 0 7 73 167 3 +270.57 ± 40.25
203907 ncm-dbt-06 1211159 500 206 53 241 +109.83 ± 14.84 0 11 83 148 8 +230.16 ± 37.67

Commit

Commit ID 70ba9de85cddc5460b1ec53e0a99bee271e26ece
Author Linmiao Xu
Date 2023-09-22 17:26:16 UTC
Update NNUE architecture to SFNNv8: L1-2560 nn-ac1dbea57aa3.nnue Creating this net involved: - a 6-stage training process from scratch. The datasets used in stages 1-5 were fully minimized. - permuting L1 weights with https://github.com/official-stockfish/nnue-pytorch/pull/254 A strong epoch after each training stage was chosen for the next. The 6 stages were: ``` 1. 400 epochs, lambda 1.0, default LR and gamma UHOx2-wIsRight-multinet-dfrc-n5000 (135G) nodes5000pv2_UHO.binpack data_pv-2_diff-100_nodes-5000.binpack wrongIsRight_nodes5000pv2.binpack multinet_pv-2_diff-100_nodes-5000.binpack dfrc_n5000.binpack 2. 800 epochs, end-lambda 0.75, LR 4.375e-4, gamma 0.995, skip 12 LeelaFarseer-T78juntoaugT79marT80dec.binpack (141G) T60T70wIsRightFarseerT60T74T75T76.binpack test78-junjulaug2022-16tb7p.no-db.min.binpack test79-mar2022-16tb7p.no-db.min.binpack test80-dec2022-16tb7p.no-db.min.binpack 3. 800 epochs, end-lambda 0.725, LR 4.375e-4, gamma 0.995, skip 20 leela93-v1-dfrc99-v2-T78juntosepT80jan-v6dd-T78janfebT79aprT80aprmay.min.binpack leela93-filt-v1.min.binpack dfrc99-16tb7p-filt-v2.min.binpack test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.binpack test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack test78-janfeb2022-16tb7p.min.binpack test79-apr2022-16tb7p.min.binpack test80-apr2022-16tb7p.min.binpack test80-may2022-16tb7p.min.binpack 4. 800 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 24 leela96-dfrc99-v2-T78juntosepT79mayT80junsepnovjan-v6dd-T80mar23-v6-T60novdecT77decT78aprmayT79aprT80may23.min.binpack leela96-filt-v2.min.binpack dfrc99-16tb7p-filt-v2.min.binpack test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.binpack test79-may2022-16tb7p.filter-v6-dd.min.binpack test80-jun2022-16tb7p.filter-v6-dd.min.binpack test80-sep2022-16tb7p.filter-v6-dd.min.binpack test80-nov2022-16tb7p.filter-v6-dd.min.binpack test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack test80-mar2023-2tb7p.v6-sk16.min.binpack test60-novdec2021-16tb7p.min.binpack test77-dec2021-16tb7p.min.binpack test78-aprmay2022-16tb7p.min.binpack test79-apr2022-16tb7p.min.binpack test80-may2023-2tb7p.min.binpack 5. 960 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28 Increased max-epoch to 960 near the end of the first 800 epochs 5af11540bbfe dataset: https://github.com/official-stockfish/Stockfish/pull/4635 6. 1000 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28 Increased max-epoch to 1000 near the end of the first 800 epochs 1ee1aba5ed dataset: https://github.com/official-stockfish/Stockfish/pull/4782 ``` L1 weights permuted with: ```bash python3 serialize.py $nnue $nnue_permuted \ --features=HalfKAv2_hm \ --ft_optimize \ --ft_optimize_data=/data/fishpack32.binpack \ --ft_optimize_count=10000 ``` Speed measurements from 100 bench runs at depth 13 with profile-build x86-64-avx2: ``` sf_base = 1329051 +/- 2224 (95%) sf_test = 1163344 +/- 2992 (95%) diff = -165706 +/- 4913 (95%) speedup = -12.46807% +/- 0.370% (95%) ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move (vs. L1-2048 nn-1ee1aba5ed4c.nnue) ep959 : 16.2 +/- 2.3 Failed 10+0.1 STC: https://tests.stockfishchess.org/tests/view/6501beee2cd016da89abab21 LLR: -2.92 (-2.94,2.94) <0.00,2.00> Total: 13184 W: 3285 L: 3535 D: 6364 Ptnml(0-2): 85, 1662, 3334, 1440, 71 Failed 180+1.8 VLTC: https://tests.stockfishchess.org/tests/view/6505cf9a72620bc881ea908e LLR: -2.94 (-2.94,2.94) <0.00,2.00> Total: 64248 W: 16224 L: 16374 D: 31650 Ptnml(0-2): 26, 6788, 18640, 6650, 20 Passed 60+0.6 th 8 VLTC SMP (STC bounds): https://tests.stockfishchess.org/tests/view/65084a4618698b74c2e541dc LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 90630 W: 23372 L: 23033 D: 44225 Ptnml(0-2): 13, 8490, 27968, 8833, 11 Passed 60+0.6 th 8 VLTC SMP: https://tests.stockfishchess.org/tests/view/6501d45d2cd016da89abacdb LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 137804 W: 35764 L: 35276 D: 66764 Ptnml(0-2): 31, 13006, 42326, 13522, 17 closes https://github.com/official-stockfish/Stockfish/pull/4795 bench 1246812
Copyright 2011–2024 Next Chess Move LLC