Dev Builds » 20230531-0651

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:55:17 583845 4002 1318 795 1889 +45.67 ± 5.15 2 232 1019 737 11 +91.29 ± 10.63
ncm-dbt-02 06:55:16 585482 4000 1305 740 1955 +49.41 ± 5.05 1 211 1016 766 6 +99.95 ± 10.63
ncm-dbt-03 06:56:42 584031 3998 1342 736 1920 +53.07 ± 5.07 2 194 1010 782 11 +107.02 ± 10.65
ncm-dbt-04 06:56:20 569411 4000 1283 742 1975 +47.28 ± 5.03 0 220 1023 753 4 +95.63 ± 10.59
ncm-dbt-05 06:56:26 578418 4000 1298 711 1991 +51.36 ± 5.05 1 204 1009 779 7 +103.92 ± 10.67
20000 6546 3724 9730 +49.35 ± 2.27 6 1061 5077 3817 39 +99.54 ± 4.75

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
427239 ncm-dbt-01 584706 2 1 1 0 0.0 ± 34.02 0 0 1 0 0 0.0 ± 30.47 ↓
427238 ncm-dbt-03 583154 498 159 96 243 +44.19 ± 14.65 0 32 122 95 0 +89.86 ± 30.89 ↓
427237 ncm-dbt-04 568355 500 167 96 237 +49.67 ± 13.97 0 25 129 96 0 +101.46 ± 29.84 ↓
427236 ncm-dbt-02 584495 500 166 100 234 +46.13 ± 13.63 0 24 136 90 0 +93.95 ± 28.85 ↓
427235 ncm-dbt-05 577069 500 166 95 239 +49.67 ± 14.38 0 27 126 96 1 +99.95 ± 30.29 ↓
427234 ncm-dbt-01 583489 500 154 109 237 +31.36 ± 15.23 1 40 122 87 0 +64.66 ± 30.91 ↓
427233 ncm-dbt-03 583279 500 173 98 229 +52.51 ± 14.39 0 26 124 99 1 +106.01 ± 30.56 ↓
427232 ncm-dbt-02 587792 500 166 86 248 +56.07 ± 14.29 0 24 123 102 1 +113.68 ± 30.67 ↓
427231 ncm-dbt-01 582527 500 158 90 252 +47.54 ± 14.66 0 28 129 90 3 +92.46 ± 29.89 ↓
427230 ncm-dbt-04 570348 500 171 97 232 +51.8 ± 14.75 0 28 122 98 2 +102.97 ± 30.86 ↓
427229 ncm-dbt-05 573729 500 149 84 267 +45.42 ± 14.14 0 26 135 87 2 +89.48 ± 29.04 ↓
427228 ncm-dbt-03 585801 500 180 96 224 +58.93 ± 13.72 0 19 129 101 1 +119.89 ± 29.69 ↓
427227 ncm-dbt-02 584706 500 156 86 258 +48.96 ± 14.74 1 28 121 100 0 +101.46 ± 31.01 ↓
427226 ncm-dbt-04 569669 500 156 95 249 +42.6 ± 15.16 0 36 118 95 1 +85.04 ± 31.44 ↓
427225 ncm-dbt-01 585548 500 165 97 238 +47.54 ± 14.4 0 28 127 94 1 +95.44 ± 30.17 ↓
427224 ncm-dbt-05 581111 500 156 77 267 +55.36 ± 13.84 0 20 133 95 2 +110.6 ± 29.15 ↓
427223 ncm-dbt-03 581652 500 172 93 235 +55.36 ± 15.06 0 29 115 104 2 +110.6 ± 31.86 ↓
427222 ncm-dbt-02 585717 500 157 89 254 +47.54 ± 13.85 0 23 138 87 2 +93.95 ± 28.55 ↓
427221 ncm-dbt-04 569031 500 161 88 251 +51.09 ± 14.85 0 31 115 104 0 +104.49 ± 31.86 ↓
427220 ncm-dbt-01 583698 500 161 106 233 +38.37 ± 14.4 0 32 132 85 1 +76.25 ± 29.54 ↓
427219 ncm-dbt-05 576987 500 169 96 235 +51.09 ± 13.76 0 22 134 93 1 +102.97 ± 29.07 ↓
427218 ncm-dbt-02 587113 500 160 89 251 +49.67 ± 14.1 0 26 127 97 0 +101.46 ± 30.14 ↓
427217 ncm-dbt-03 584159 500 162 90 248 +50.38 ± 14.28 1 22 133 92 2 +101.46 ± 29.24 ↓
427216 ncm-dbt-01 582027 500 172 89 239 +58.21 ± 14.53 0 25 118 106 1 +118.33 ± 31.4 ↓
427215 ncm-dbt-04 568196 500 157 97 246 +41.89 ± 13.94 0 28 134 88 0 +85.04 ± 29.21 ↓
427214 ncm-dbt-05 579868 500 162 88 250 +51.8 ± 14.49 0 28 120 102 0 +106.01 ± 31.14 ↓
427213 ncm-dbt-02 586139 500 167 95 238 +50.38 ± 14.68 0 28 124 96 2 +99.95 ± 30.58 ↓
427212 ncm-dbt-03 584034 500 168 84 248 +58.93 ± 13.43 0 17 133 99 1 +119.89 ± 29.04 ↓
427211 ncm-dbt-01 583279 500 162 105 233 +39.78 ± 14.35 0 32 129 89 0 +80.63 ± 29.95 ↓
427210 ncm-dbt-04 568832 500 152 78 270 +51.8 ± 14.62 0 29 118 103 0 +106.01 ± 31.43 ↓
427209 ncm-dbt-05 581735 500 172 93 235 +55.36 ± 14.39 0 26 119 105 0 +113.68 ± 31.27 ↓
427208 ncm-dbt-02 584369 500 180 107 213 +51.09 ± 14.85 0 30 118 101 1 +102.97 ± 31.44 ↓
427207 ncm-dbt-03 584285 500 173 85 242 +61.79 ± 13.85 0 19 125 105 1 +126.18 ± 30.27 ↓
427206 ncm-dbt-01 583614 500 183 96 221 +61.08 ± 14.65 1 20 123 103 3 +123.02 ± 30.61 ↓
427205 ncm-dbt-05 578877 500 159 96 245 +44.01 ± 14.33 0 30 127 93 0 +89.48 ± 30.19 ↓
427204 ncm-dbt-04 570148 500 160 92 248 +47.54 ± 12.84 0 17 149 83 1 +95.44 ± 26.81 ↓
427203 ncm-dbt-01 585717 500 162 102 236 +41.89 ± 14.07 0 27 138 83 2 +82.1 ± 28.65 ↓
427202 ncm-dbt-02 583531 500 153 88 259 +45.42 ± 14.14 0 28 129 93 0 +92.46 ± 29.89 ↓
427201 ncm-dbt-03 585885 500 155 94 251 +42.6 ± 15.03 1 30 129 87 3 +83.57 ± 29.93 ↓
427200 ncm-dbt-04 570709 500 159 99 242 +41.89 ± 13.66 0 26 138 86 0 +85.04 ± 28.63 ↓
427199 ncm-dbt-05 577971 500 165 82 253 +58.21 ± 14.93 1 25 115 108 1 +119.89 ± 31.85 ↓

Commit

Commit ID c1fff71650e2f8bf5a2d63bdc043161cdfe8e460
Author Linmiao Xu
Date 2023-05-31 06:51:22 UTC
Update NNUE architecture to SFNNv6 with larger L1 size of 1536 Created by training a new net from scratch with L1 size increased from 1024 to 1536. Thanks to Vizvezdenec for the idea of exploring larger net sizes after recent training data improvements. A new net was first trained with lambda 1.0 and constant LR 8.75e-4. Then a strong net from a later epoch in the training run was chosen for retraining with start-lambda 1.0 and initial LR 4.375e-4 decaying with gamma 0.995. Retraining was performed a total of 3 times, for this 4-step process: 1. 400 epochs, lambda 1.0 on filtered T77+T79 v6 deduplicated data 2. 800 epochs, end-lambda 0.75 on T60T70wIsRightFarseerT60T74T75T76.binpack 3. 800 epochs, end-lambda 0.75 and early-fen-skipping 28 on the master dataset 4. 800 epochs, end-lambda 0.7 and early-fen-skipping 28 on the master dataset In the training sequence that reached the new nn-8d69132723e2.nnue net, the epochs used for the 3x retraining runs were: 1. epoch 379 trained on T77T79-filter-v6-dd.min.binpack 2. epoch 679 trained on T60T70wIsRightFarseerT60T74T75T76.binpack 3. epoch 799 trained on the master dataset For training from scratch: python3 easy_train.py \ --experiment-name new-L1-1536-T77T79-filter-v6dd \ --training-dataset /data/T77T79-filter-v6-dd.min.binpack \ --max_epoch 400 \ --lambda 1.0 \ --start-from-engine-test-net False \ --engine-test-branch linrock/Stockfish/L1-1536 \ --nnue-pytorch-branch linrock/Stockfish/misc-fixes-L1-1536 \ --tui False \ --gpus "0," \ --seed $RANDOM Retraining commands were similar to each other. For the 3rd retraining run: python3 easy_train.py \ --experiment-name L1-1536-T77T79-v6dd-Re1-LeelaFarseer-Re2-masterDataset-Re3-sameData \ --training-dataset /data/leela96-dfrc99-v2-T60novdecT80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack \ --early-fen-skipping 28 \ --max_epoch 800 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-T77T79-v6dd-Re1-LeelaFarseer-Re2-masterDataset-nn-epoch799.nnue \ --engine-test-branch linrock/Stockfish/L1-1536 \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --tui False \ --gpus "0," \ --seed $RANDOM The T77+T79 data used is a subset of the master dataset available at: https://robotmoon.com/nnue-training-data/ T60T70wIsRightFarseerT60T74T75T76.binpack is available at: https://drive.google.com/drive/folders/1S9-ZiQa_3ApmjBtl2e8SyHxj4zG4V8gG Local elo at 25k nodes per move vs. nn-e1fb1ade4432.nnue (L1 size 1024): nn-epoch759.nnue : 26.9 +/- 1.6 Failed STC https://tests.stockfishchess.org/tests/view/64742485d29264e4cfa75f97 LLR: -2.94 (-2.94,2.94) <0.00,2.00> Total: 13728 W: 3588 L: 3829 D: 6311 Ptnml(0-2): 71, 1661, 3610, 1482, 40 Failing LTC https://tests.stockfishchess.org/tests/view/64752d7c4a36543c4c9f3618 LLR: -1.91 (-2.94,2.94) <0.50,2.50> Total: 35424 W: 9522 L: 9603 D: 16299 Ptnml(0-2): 24, 3579, 10585, 3502, 22 Passed VLTC 180+1.8 https://tests.stockfishchess.org/tests/view/64752df04a36543c4c9f3638 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 47616 W: 13174 L: 12863 D: 21579 Ptnml(0-2): 13, 4261, 14952, 4566, 16 Passed VLTC SMP 60+0.6 th 8 https://tests.stockfishchess.org/tests/view/647446ced29264e4cfa761e5 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 19942 W: 5694 L: 5451 D: 8797 Ptnml(0-2): 6, 1504, 6707, 1749, 5 closes https://github.com/official-stockfish/Stockfish/pull/4593 bench 2222567
Copyright 2011–2025 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version