Dev Builds » 20240601-1759

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:45:27 583714 4000 1504 615 1881 +78.53 ± 5.03 2 123 878 978 19 +162.35 ± 11.46
ncm-dbt-02 06:48:12 586001 4000 1535 609 1856 +81.92 ± 4.91 3 102 874 1008 13 +171.9 ± 11.46
ncm-dbt-03 06:48:46 586862 4014 1486 620 1908 +76.15 ± 4.92 0 122 911 960 14 +157.45 ± 11.22
ncm-dbt-04 06:48:13 571050 4002 1529 620 1853 +80.32 ± 4.87 1 105 891 992 12 +167.87 ± 11.34
ncm-dbt-05 06:47:36 584052 3984 1513 576 1895 +83.27 ± 5.0 3 107 850 1014 18 +174.04 ± 11.64
20000 7567 3040 9393 +80.03 ± 2.21 9 559 4404 4952 76 +166.66 ± 5.11

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
388074 ncm-dbt-04 571310 2 1 1 0 0.0 ± 34.02 0 0 1 0 0 0.0 ± 30.47
388073 ncm-dbt-03 586350 14 5 1 8 +102.04 ± 70.73 0 0 3 4 0 +225.59 ± 263.0
388072 ncm-dbt-05 587707 484 191 62 231 +94.89 ± 12.88 0 5 104 132 1 +204.52 ± 32.94
388071 ncm-dbt-02 585042 500 200 67 233 +94.69 ± 13.78 0 10 100 137 3 +200.24 ± 34.09
388070 ncm-dbt-04 570068 500 186 76 238 +77.7 ± 13.09 0 9 124 115 2 +160.64 ± 29.94
388069 ncm-dbt-01 581943 500 171 65 264 +74.79 ± 13.62 0 14 117 118 1 +155.54 ± 31.31
388068 ncm-dbt-03 586139 500 192 87 221 +74.06 ± 13.74 0 13 122 112 3 +150.51 ± 30.49
388067 ncm-dbt-05 585632 500 180 87 233 +65.38 ± 14.29 0 20 119 109 2 +132.54 ± 31.18
388066 ncm-dbt-02 586646 500 189 77 234 +79.17 ± 13.14 0 10 119 120 1 +165.8 ± 30.8
388065 ncm-dbt-04 569949 500 191 80 229 +78.44 ± 13.27 0 12 115 123 0 +165.8 ± 31.55
388064 ncm-dbt-01 583363 500 189 81 230 +76.25 ± 14.81 1 18 105 124 2 +158.93 ± 33.37
388063 ncm-dbt-03 583782 500 173 69 258 +73.33 ± 13.86 0 16 115 118 1 +152.18 ± 31.7
388062 ncm-dbt-05 579702 500 197 71 232 +89.48 ± 13.53 0 10 106 132 2 +189.0 ± 32.98
388061 ncm-dbt-02 584411 500 179 72 249 +75.52 ± 14.51 1 16 110 121 2 +157.24 ± 32.52
388060 ncm-dbt-03 586646 500 204 81 215 +87.26 ± 13.33 0 9 111 128 2 +183.51 ± 32.06
388059 ncm-dbt-01 582402 500 192 56 252 +96.94 ± 13.96 0 11 95 141 3 +206.01 ± 35.09
388058 ncm-dbt-04 573446 500 182 83 235 +69.71 ± 13.88 0 17 118 114 1 +143.89 ± 31.26
388057 ncm-dbt-05 587749 500 194 73 233 +85.78 ± 14.51 0 14 106 125 5 +174.55 ± 33.13
388056 ncm-dbt-02 585042 500 190 80 230 +77.71 ± 14.15 1 14 110 124 1 +164.07 ± 32.48
388055 ncm-dbt-03 586943 500 190 81 229 +76.98 ± 13.98 0 17 107 126 0 +162.35 ± 33.02
388054 ncm-dbt-01 582694 500 183 87 230 +67.54 ± 14.37 0 20 116 112 2 +137.37 ± 31.63
388053 ncm-dbt-04 571672 500 204 73 223 +93.2 ± 13.28 0 9 102 138 1 +200.24 ± 33.67
388052 ncm-dbt-05 581860 500 184 77 239 +75.52 ± 14.79 2 15 109 122 2 +158.93 ± 32.69
388051 ncm-dbt-02 588089 500 194 77 229 +82.83 ± 14.29 0 16 103 129 2 +172.78 ± 33.69
388050 ncm-dbt-04 568673 500 189 79 232 +77.7 ± 14.29 0 17 108 123 2 +160.64 ± 32.85
388049 ncm-dbt-03 588047 500 190 77 233 +79.9 ± 14.5 0 18 103 127 2 +165.8 ± 33.7
388048 ncm-dbt-01 581652 500 193 84 223 +76.97 ± 14.56 0 17 111 118 4 +155.54 ± 32.36
388047 ncm-dbt-05 586054 500 183 69 248 +80.63 ± 14.52 0 18 102 128 2 +167.53 ± 33.88
388046 ncm-dbt-04 570909 500 196 64 240 +93.95 ± 13.61 0 9 103 135 3 +198.34 ± 33.48
388045 ncm-dbt-01 585885 500 193 86 221 +75.52 ± 14.65 0 19 108 120 3 +153.86 ± 32.88
388044 ncm-dbt-02 587579 500 206 82 212 +88.0 ± 13.66 0 12 103 134 1 +187.16 ± 33.6
388043 ncm-dbt-03 586689 500 177 71 252 +74.79 ± 13.62 0 13 120 115 2 +153.86 ± 30.8
388042 ncm-dbt-05 578671 500 183 77 240 +74.79 ± 14.34 1 17 107 125 0 +158.93 ± 33.03
388041 ncm-dbt-01 584748 500 193 74 233 +84.31 ± 14.18 1 12 106 129 2 +178.11 ± 33.1
388040 ncm-dbt-04 571029 500 186 94 220 +64.66 ± 14.53 1 18 122 106 3 +130.94 ± 30.71
388039 ncm-dbt-03 589753 500 177 72 251 +74.06 ± 13.59 0 13 121 114 2 +152.18 ± 30.64
388038 ncm-dbt-02 582903 500 191 88 221 +72.61 ± 13.54 1 11 123 114 1 +152.18 ± 30.28
388037 ncm-dbt-01 587028 500 190 82 228 +76.25 ± 13.51 0 12 120 116 2 +157.24 ± 30.75
388036 ncm-dbt-05 585042 500 201 60 239 +100.7 ± 13.69 0 8 97 141 4 +213.85 ± 34.58
388035 ncm-dbt-02 588302 500 186 66 248 +85.04 ± 13.9 0 13 106 129 2 +178.11 ± 33.1
388034 ncm-dbt-04 572396 500 194 70 236 +88.0 ± 13.81 0 14 98 138 0 +189.0 ± 34.56
388033 ncm-dbt-03 587409 500 178 81 241 +68.27 ± 14.81 0 23 109 116 2 +138.99 ± 32.74

Commit

Commit ID cb4a62311985f685ba6f5457851527a3289073e6
Author Linmiao Xu
Date 2024-06-01 17:59:07 UTC
Update default smallnet to nn-37f18f62d772.nnue Created by training L1-128 from scratch with: - skipping based on simple eval in the trainer, for compatibility with regular binpacks without requiring pre-filtering all binpacks - minimum simple eval of 950, lower than 1000 previously - usage of some hse-v1 binpacks with minimum simple eval 1000 - addition of hse-v6 binpacks with minimum simple eval 500 - permuting the FT with 10k positions from fishpack32.binpack - torch.compile to speed up smallnet training Training is significantly slower when using non-pre-filtered binpacks due to the increased skipping required. This net was reached at epoch 339. ``` experiment-name: 128--S1-hse-1k-T80-v6-unfilt-less-sf--se-gt950-no-wld-skip training-dataset: /data/: - dfrc99-16tb7p.v2.min.binpack /data/hse-v1/: - leela96-filt-v2.min.high-simple-eval-1k.min-v2.binpack - test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test77-nov2021-2tb7p.no-db.min.high-simple-eval-1k.min-v2.binpack - test77-dec2021-16tb7p.no-db.min.high-simple-eval-1k.min-v2.binpack - test77-jan2022-2tb7p.high-simple-eval-1k.min-v2.binpack - test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test79-apr2022-16tb7p.min.high-simple-eval-1k.min-v2.binpack - test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test80-apr2022-16tb7p.min.high-simple-eval-1k.min-v2.binpack - test80-may2022-16tb7p.high-simple-eval-1k.min-v2.binpack - test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.min-v2.binpack - test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack - test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.min-v2.binpack /data/S11-mar2024/: - test80-2022-08-aug-16tb7p.v6-dd.min.binpack - test80-2022-10-oct-16tb7p.v6-dd.binpack - test80-2022-12-dec-16tb7p.min.binpack - test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - test80-2023-02-feb-16tb7p.v6-sk20.min.binpack - test80-2023-03-mar-2tb7p.v6-sk16.min.binpack - test80-2023-04-apr-2tb7p.v6-sk16.min.binpack - test80-2023-05-may-2tb7p.v6.min.binpack - test80-2023-06-jun-2tb7p.binpack.min-v2.binpack - test80-2023-07-jul-2tb7p.binpack.min-v2.binpack - test80-2023-08-aug-2tb7p.v6.min.binpack - test80-2023-09-sep-2tb7p.binpack.hse-v6.binpack - test80-2023-10-oct-2tb7p.binpack.hse-v6.binpack - test80-2023-11-nov-2tb7p.binpack.hse-v6.binpack - test80-2023-12-dec-2tb7p.binpack.hse-v6.binpack - test80-2024-01-jan-2tb7p.binpack.hse-v6.binpack - test80-2024-02-feb-2tb7p.binpack.hse-v6.binpack - test80-2024-03-mar-2tb7p.binpack wld-fen-skipping: False nnue-pytorch-branch: linrock/nnue-pytorch/128-skipSimpleEval-lt950-torch-compile engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 start-from-engine-test-net: False num-epochs: 500 start-lambda: 1.0 end-lambda: 1.0 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Passed STC: https://tests.stockfishchess.org/tests/view/66549c16a86388d5e27daff5 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 196608 W: 51254 L: 50697 D: 94657 Ptnml(0-2): 722, 23244, 49796, 23839, 703 Passed LTC: https://tests.stockfishchess.org/tests/view/6658d1aa6b0e318cefa90122 LLR: 2.96 (-2.94,2.94) <0.50,2.50> Total: 122538 W: 31332 L: 30835 D: 60371 Ptnml(0-2): 69, 13407, 33811, 13922, 60 closes https://github.com/official-stockfish/Stockfish/pull/5333 bench
Copyright 2011–2024 Next Chess Move LLC