Dev Builds » 20240107-2015

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:48:44 1202631 3328 1447 203 1678 +136.48 ± 5.13 0 31 403 1185 45 +315.75 ± 16.99
ncm-dbt-02 09:44:51 1235900 3310 1447 226 1637 +134.5 ± 5.16 0 25 434 1146 50 +306.53 ± 16.35
ncm-dbt-03 09:47:25 1183727 3340 1466 242 1632 +133.53 ± 5.21 1 30 431 1160 48 +304.62 ± 16.43
ncm-dbt-04 09:49:37 1218787 3346 1433 220 1693 +131.95 ± 5.07 0 34 425 1181 33 +304.98 ± 16.55
ncm-dbt-05 09:44:14 1230986 3324 1444 252 1628 +130.39 ± 5.15 0 24 469 1122 47 +293.9 ± 15.7
ncm-dbt-06 09:48:23 1227575 3352 1463 239 1650 +133.01 ± 5.13 0 32 430 1172 42 +304.93 ± 16.45
20000 8700 1382 9918 +133.3 ± 2.1 1 176 2592 6966 265 +305.04 ± 6.69

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
244379 ncm-dbt-03 1224808 284 120 15 149 +134.83 ± 17.45 0 3 34 102 3 +314.13 ± 59.86
244378 ncm-dbt-02 1229570 310 135 17 158 +139.25 ± 16.7 0 0 44 104 7 +312.56 ± 51.57
244377 ncm-dbt-05 1240826 324 140 19 165 +136.34 ± 15.96 0 0 47 109 6 +308.15 ± 49.79
244376 ncm-dbt-01 1199086 328 146 21 161 +139.44 ± 16.44 0 5 32 124 3 +333.25 ± 61.36
244375 ncm-dbt-04 1208271 346 135 25 186 +114.42 ± 15.5 0 4 56 112 1 +257.63 ± 45.96
244374 ncm-dbt-06 1224298 352 148 28 176 +123.38 ± 16.13 0 5 49 119 3 +278.41 ± 49.4
244373 ncm-dbt-03 823359 500 225 31 244 +142.25 ± 13.75 0 4 59 176 11 +324.17 ± 44.97
244372 ncm-dbt-02 1223675 500 217 31 252 +135.76 ± 13.39 0 4 64 174 8 +309.64 ± 43.08
244371 ncm-dbt-05 1210612 500 222 41 237 +131.74 ± 13.12 0 2 73 167 8 +295.94 ± 39.99
244370 ncm-dbt-01 1202771 500 224 29 247 +143.07 ± 13.34 0 5 53 184 8 +336.46 ± 47.52
244369 ncm-dbt-04 1212758 500 205 41 254 +118.33 ± 13.35 0 7 75 165 3 +265.78 ± 39.69
244368 ncm-dbt-06 1222738 500 212 35 253 +128.55 ± 13.56 0 6 67 171 6 +290.66 ± 42.08
244367 ncm-dbt-03 1245739 56 22 4 30 +115.77 ± 39.61 0 1 8 19 0 +265.1 ± 133.83
244366 ncm-dbt-02 1233094 500 219 32 249 +136.56 ± 12.59 0 2 65 177 6 +318.25 ± 42.59
244365 ncm-dbt-05 1214218 500 221 44 235 +128.55 ± 13.91 0 6 69 167 8 +285.49 ± 41.44
244364 ncm-dbt-01 1230080 500 223 28 249 +143.07 ± 13.54 0 4 57 179 10 +330.23 ± 45.79
244363 ncm-dbt-04 1216603 500 225 20 255 +151.34 ± 11.91 0 2 47 195 6 +377.87 ± 50.67
244362 ncm-dbt-03 1253077 500 231 40 229 +139.81 ± 12.47 0 1 64 178 7 +327.18 ± 42.83
244361 ncm-dbt-06 1243645 500 217 37 246 +130.94 ± 12.95 0 5 64 177 4 +304.07 ± 43.1
244360 ncm-dbt-02 1224779 500 219 31 250 +137.37 ± 13.53 0 5 60 177 8 +315.35 ± 44.57
244359 ncm-dbt-05 1218978 500 215 48 237 +120.67 ± 14.18 0 7 77 158 8 +261.07 ± 39.14
244358 ncm-dbt-01 1181130 500 218 25 257 +141.44 ± 13.4 0 5 55 182 8 +330.23 ± 46.62
244357 ncm-dbt-04 1228041 500 221 41 238 +130.94 ± 13.51 0 5 67 171 7 +295.94 ± 42.08
244356 ncm-dbt-06 1226433 500 212 29 259 +133.34 ± 12.09 0 2 66 179 3 +315.35 ± 42.24
244355 ncm-dbt-03 1245589 500 222 41 237 +131.74 ± 13.67 0 6 64 173 7 +298.62 ± 43.1
244354 ncm-dbt-01 1224217 500 212 31 257 +131.74 ± 12.74 0 3 68 174 5 +304.07 ± 41.65
244353 ncm-dbt-05 1241059 500 217 27 256 +138.99 ± 13.67 0 3 65 171 11 +312.48 ± 42.67
244352 ncm-dbt-02 1241293 500 222 43 235 +130.14 ± 13.52 0 3 74 164 9 +288.06 ± 39.79
244351 ncm-dbt-04 1208507 500 225 39 236 +135.76 ± 13.39 0 7 55 183 5 +318.25 ± 46.48
244350 ncm-dbt-06 1229565 500 223 33 244 +138.99 ± 13.29 0 4 60 178 8 +321.19 ± 44.57
244349 ncm-dbt-03 1204919 500 221 44 235 +128.55 ± 14.42 1 7 63 172 7 +290.66 ± 43.37
244348 ncm-dbt-01 1215044 500 210 31 259 +130.14 ± 12.59 0 3 69 174 4 +301.33 ± 41.33
244347 ncm-dbt-05 1245111 500 214 33 253 +131.74 ± 12.34 0 4 63 181 2 +312.48 ± 43.44
244346 ncm-dbt-02 1245991 500 222 35 243 +136.56 ± 13.73 0 5 62 174 9 +309.64 ± 43.82
244345 ncm-dbt-04 1240189 500 208 30 262 +129.35 ± 13.17 0 3 73 167 7 +290.66 ± 40.08
244344 ncm-dbt-06 1236647 500 225 39 236 +135.76 ± 14.12 0 5 65 169 11 +301.33 ± 42.75
244343 ncm-dbt-03 1246144 500 228 29 243 +146.36 ± 13.99 0 5 53 180 12 +336.46 ± 47.52
244342 ncm-dbt-05 1246101 500 215 40 245 +126.97 ± 12.47 0 2 75 169 4 +290.66 ± 39.4
244341 ncm-dbt-02 1252900 500 213 37 250 +127.76 ± 13.02 0 6 65 176 3 +295.94 ± 42.75
244340 ncm-dbt-06 1209705 500 226 38 236 +137.37 ± 13.34 0 5 59 179 7 +318.25 ± 44.96
244339 ncm-dbt-01 1166093 500 214 38 248 +127.76 ± 13.75 0 6 69 168 7 +285.49 ± 41.44
244338 ncm-dbt-03 1226184 500 197 38 265 +114.45 ± 12.3 0 3 86 160 1 +258.75 ± 36.62
244337 ncm-dbt-04 1217142 500 214 24 262 +138.99 ± 12.9 0 6 52 188 4 +333.32 ± 47.9

Commit

Commit ID 584d9efedcde330eeb96a99215552ddfb06f52ba
Author Linmiao Xu
Date 2024-01-07 20:15:52 UTC
Dual NNUE with L1-128 smallnet Credit goes to @mstembera for: - writing the code enabling dual NNUE: https://github.com/official-stockfish/Stockfish/pull/4898 - the idea of trying L1-128 trained exclusively on high simple eval positions The L1-128 smallnet is: - epoch 399 of a single-stage training from scratch - trained only on positions from filtered data with high material difference - defined by abs(simple_eval) > 1000 ```yaml experiment-name: 128--S1-only-hse-v2 training-dataset: - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-1k.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack # T80 2022 - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-1k.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.binpack # T80 2023 - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-1k.binpack start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: https://github.com/official-stockfish/nnue-pytorch/pull/259 Data filtered for high simple eval positions with: https://github.com/linrock/nnue-data/blob/32d6a68/filter_high_simple_eval_plain.py https://github.com/linrock/Stockfish/blob/61dbfe/src/tools/transform.cpp#L626-L655 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch399.nnue : -318.1 +/- 2.1 Passed STC: https://tests.stockfishchess.org/tests/view/6574cb9d95ea6ba1fcd49e3b LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 62432 W: 15875 L: 15521 D: 31036 Ptnml(0-2): 177, 7331, 15872, 7633, 203 Passed LTC: https://tests.stockfishchess.org/tests/view/6575da2d4d789acf40aaac6e LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64830 W: 16118 L: 15738 D: 32974 Ptnml(0-2): 43, 7129, 17697, 7497, 49 closes https://github.com/official-stockfish/Stockfish/pulls Bench: 1330050 Co-Authored-By: mstembera <5421953+mstembera@users.noreply.github.com>
Copyright 2011–2024 Next Chess Move LLC