Dev Builds » 20240107-2020

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:53:03 577923 4016 1433 605 1978 +72.67 ± 5.05 3 147 884 967 7 +151.5 ± 11.44
ncm-dbt-02 06:53:02 585496 3984 1410 591 1983 +72.46 ± 4.96 0 134 916 931 11 +149.52 ± 11.2
ncm-dbt-03 06:51:43 583505 4000 1442 588 1970 +75.34 ± 4.88 1 114 927 946 12 +156.18 ± 11.1
ncm-dbt-04 06:52:59 569027 4000 1450 635 1915 +71.79 ± 4.94 1 136 917 939 7 +149.06 ± 11.2
ncm-dbt-05 06:53:41 581927 4000 1398 635 1967 +67.1 ± 5.0 2 150 939 901 8 +138.39 ± 11.07
20000 7133 3054 9813 +71.87 ± 2.22 7 681 4583 4684 45 +148.89 ± 5.01

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
420186 ncm-dbt-01 578918 16 5 3 8 +43.62 ± 53.54 0 0 6 2 0 +88.68 ± 116.18
420185 ncm-dbt-02 584453 484 174 78 232 +69.84 ± 14.5 0 19 109 113 1 +144.13 ± 32.68
420184 ncm-dbt-05 581277 500 181 89 230 +64.66 ± 14.39 0 22 115 112 1 +132.54 ± 31.82
420183 ncm-dbt-03 581402 500 189 78 233 +78.44 ± 14.46 1 14 111 121 3 +162.35 ± 32.31
420182 ncm-dbt-04 568593 500 189 73 238 +82.1 ± 14.13 0 16 103 130 1 +172.78 ± 33.69
420181 ncm-dbt-01 577438 500 178 72 250 +74.79 ± 15.04 2 19 100 129 0 +160.64 ± 34.22
420180 ncm-dbt-02 585084 500 175 86 239 +62.51 ± 13.88 0 20 121 109 0 +129.35 ± 30.88
420179 ncm-dbt-03 582193 500 193 72 235 +85.78 ± 13.46 0 10 111 127 2 +179.9 ± 32.11
420178 ncm-dbt-05 577725 500 175 99 226 +53.22 ± 13.73 0 22 130 98 0 +109.07 ± 29.63
420177 ncm-dbt-04 568991 500 176 95 229 +56.78 ± 13.91 0 20 131 97 2 +113.68 ± 29.44
420176 ncm-dbt-01 575394 500 184 66 250 +83.57 ± 13.56 0 13 106 131 0 +178.11 ± 33.1
420175 ncm-dbt-02 584201 500 173 72 255 +71.16 ± 13.79 0 15 121 112 2 +145.54 ± 30.73
420174 ncm-dbt-05 580945 500 184 69 247 +81.36 ± 13.96 0 15 106 128 1 +171.02 ± 33.15
420173 ncm-dbt-03 585506 500 180 78 242 +71.88 ± 14.1 0 16 119 112 3 +145.54 ± 31.07
420172 ncm-dbt-04 569430 500 173 68 259 +74.06 ± 14.03 0 18 109 123 0 +155.54 ± 32.7
420171 ncm-dbt-01 577479 500 177 78 245 +69.71 ± 13.29 0 13 126 110 1 +143.89 ± 29.88
420170 ncm-dbt-02 586520 500 176 76 248 +70.43 ± 13.76 0 15 122 111 2 +143.89 ± 30.58
420169 ncm-dbt-05 586816 500 164 80 256 +58.93 ± 14.29 1 20 124 104 1 +121.46 ± 30.47
420168 ncm-dbt-03 586181 500 185 67 248 +83.57 ± 13.09 0 10 112 128 0 +178.11 ± 31.94
420167 ncm-dbt-04 568951 500 191 83 226 +76.25 ± 14.39 1 16 108 124 1 +160.64 ± 32.85
420166 ncm-dbt-01 580157 500 185 77 238 +76.25 ± 13.96 0 16 111 122 1 +158.93 ± 32.34
420165 ncm-dbt-05 584327 500 181 81 238 +70.44 ± 14.89 1 21 106 121 1 +147.19 ± 33.22
420164 ncm-dbt-02 585042 500 182 62 256 +85.04 ± 13.6 0 11 110 127 2 +178.11 ± 32.33
420163 ncm-dbt-03 585169 500 176 76 248 +70.43 ± 14.2 0 18 116 114 2 +143.89 ± 31.59
420162 ncm-dbt-04 569989 500 177 84 239 +65.38 ± 13.57 0 17 123 110 0 +135.76 ± 30.5
420161 ncm-dbt-01 578753 500 181 83 236 +68.99 ± 14.28 0 20 113 116 1 +142.26 ± 32.1
420160 ncm-dbt-03 581444 500 170 69 261 +71.16 ± 14.36 0 20 110 119 1 +147.19 ± 32.57
420159 ncm-dbt-05 581652 500 174 75 251 +69.71 ± 13.88 0 16 121 111 2 +142.26 ± 30.77
420158 ncm-dbt-02 586223 500 184 80 236 +73.33 ± 13.86 0 16 115 118 1 +152.18 ± 31.7
420157 ncm-dbt-01 577725 500 167 77 256 +63.23 ± 14.88 0 24 115 108 3 +126.18 ± 31.84
420156 ncm-dbt-04 568395 500 192 73 235 +84.31 ± 12.62 0 7 117 126 0 +179.9 ± 30.91
420155 ncm-dbt-03 581902 500 176 78 246 +68.99 ± 12.96 0 11 131 107 1 +142.26 ± 29.02
420154 ncm-dbt-02 584958 500 176 71 253 +74.06 ± 13.89 0 17 111 122 0 +155.54 ± 32.36
420153 ncm-dbt-05 581652 500 162 70 268 +64.66 ± 13.54 0 17 124 109 0 +134.15 ± 30.35
420152 ncm-dbt-01 577848 500 168 79 253 +62.51 ± 14.72 1 22 115 111 1 +129.35 ± 31.83
420151 ncm-dbt-04 567522 500 172 76 252 +67.54 ± 14.78 0 23 110 115 2 +137.37 ± 32.59
420150 ncm-dbt-03 584243 500 173 70 257 +72.61 ± 13.54 0 15 117 118 0 +152.18 ± 31.35
420149 ncm-dbt-02 587494 500 170 66 264 +73.33 ± 14.85 0 21 107 119 3 +148.85 ± 33.05
420148 ncm-dbt-05 581028 500 177 72 251 +74.06 ± 14.18 0 17 113 118 2 +152.18 ± 32.04
420147 ncm-dbt-01 577602 500 188 70 242 +83.57 ± 14.6 0 20 92 138 0 +178.11 ± 35.66
420146 ncm-dbt-04 570348 500 180 83 237 +68.27 ± 14.12 0 19 116 114 1 +140.62 ± 31.61

Commit

Commit ID f09adaa4a4c3cbb44e1ca8cc687a08dc3d58076e
Author Linmiao Xu
Date 2024-01-07 20:20:15 UTC
Update smallnet to nn-baff1ede1f90.nnue with wider eval range Created by training an L1-128 net from scratch with a wider range of evals in the training data and wld-fen-skipping disabled during training. The differences in this training data compared to the first dual nnue PR are: - removal of all positions with 3 pieces - when piece count >= 16, keep positions with simple eval above 750 - when piece count < 16, remove positions with simple eval above 3000 The asymmetric data filtering was meant to flatten the training data piece count distribution, which was previously heavily skewed towards positions with low piece counts. Additionally, the simple eval range where the smallnet is used was widened to cover more positions previously evaluated by the big net and simple eval. ```yaml experiment-name: 128--S1-hse-S7-v4-S3-v1-no-wld-skip training-dataset: - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-v4.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-v4.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-v4.binpack wld-fen-skipping: False start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 start-lambda: 1.0 end-lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: https://github.com/official-stockfish/nnue-pytorch/pull/259 FT weights permuted with 10k positions from fishpack32.binpack with: https://github.com/official-stockfish/nnue-pytorch/pull/254 Data filtered for high simple eval positions (v4) with: https://github.com/linrock/Stockfish/blob/b9c8440/src/tools/transform.cpp#L640-L675 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch319.nnue : -241.7 +/- 3.2 Passed STC vs. 36db936: https://tests.stockfishchess.org/tests/view/6576b3484d789acf40aabbfe LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 21920 W: 5680 L: 5381 D: 10859 Ptnml(0-2): 82, 2488, 5520, 2789, 81 Passed LTC vs. DualNNUE #4915: https://tests.stockfishchess.org/tests/view/65775c034d789acf40aac7e3 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 147606 W: 36619 L: 36063 D: 74924 Ptnml(0-2): 98, 16591, 39891, 17103, 120 closes https://github.com/official-stockfish/Stockfish/pull/4919 Bench: 1438336
Copyright 2011–2025 Next Chess Move LLC