Dev Builds » 20240709-1649

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:54:16 584636 4000 1510 582 1908 +82.1 ± 4.94 1 112 858 1016 13 +171.9 ± 11.59
ncm-dbt-02 06:53:21 588233 4000 1494 573 1933 +81.46 ± 5.01 1 124 841 1021 13 +170.37 ± 11.73
ncm-dbt-03 06:52:56 588105 4000 1519 558 1923 +85.14 ± 4.91 1 107 831 1052 9 +180.13 ± 11.79
ncm-dbt-04 06:54:02 557752 4000 1505 586 1909 +81.27 ± 4.94 2 116 851 1023 8 +171.24 ± 11.65
ncm-dbt-05 06:53:32 581964 4000 1446 597 1957 +74.88 ± 4.88 2 115 924 950 9 +155.97 ± 11.12
20000 7474 2896 9630 +80.96 ± 2.21 7 574 4305 5062 52 +169.84 ± 5.17

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
374433 ncm-dbt-01 587367 500 190 67 243 +87.26 ± 13.95 0 13 103 132 2 +183.51 ± 33.63 ↓
374432 ncm-dbt-05 578013 500 178 72 250 +74.79 ± 13.31 1 9 124 115 1 +157.24 ± 30.01 ↓
374431 ncm-dbt-04 558055 500 189 76 235 +79.9 ± 14.21 0 18 101 131 0 +169.27 ± 34.05 ↓
374430 ncm-dbt-03 587877 500 182 77 241 +74.06 ± 14.03 0 18 109 123 0 +155.54 ± 32.7 ↓
374429 ncm-dbt-02 589027 500 187 58 255 +91.71 ± 13.26 0 9 104 136 1 +196.45 ± 33.3 ↓
374428 ncm-dbt-01 586266 500 175 63 262 +79.17 ± 13.75 0 12 117 118 3 +162.35 ± 31.23 ↓
374427 ncm-dbt-03 588728 500 183 70 247 +79.9 ± 14.5 0 18 103 127 2 +165.8 ± 33.7 ↓
374426 ncm-dbt-02 586350 500 200 71 229 +91.71 ± 14.63 0 16 92 139 3 +192.71 ± 35.72 ↓
374425 ncm-dbt-04 555880 500 191 75 234 +82.1 ± 13.52 0 12 111 126 1 +172.78 ± 32.21 ↓
374424 ncm-dbt-05 583447 500 193 67 240 +89.48 ± 14.0 1 8 109 128 4 +187.16 ± 32.4 ↓
374423 ncm-dbt-01 583154 500 189 88 223 +71.16 ± 14.36 0 21 107 122 0 +148.85 ± 33.05 ↓
374422 ncm-dbt-05 581860 500 170 78 252 +64.66 ± 13.39 0 16 126 108 0 +134.15 ± 30.02 ↓
374421 ncm-dbt-04 559014 500 178 70 252 +76.25 ± 13.96 0 17 108 125 0 +160.64 ± 32.85 ↓
374420 ncm-dbt-03 587919 500 189 64 247 +88.74 ± 13.98 0 13 101 134 2 +187.16 ± 33.99 ↓
374419 ncm-dbt-02 588387 500 174 81 245 +65.38 ± 15.1 0 24 113 109 4 +129.35 ± 32.14 ↓
374418 ncm-dbt-05 583196 500 181 70 249 +78.44 ± 13.58 0 14 111 125 0 +165.8 ± 32.28 ↓
374417 ncm-dbt-01 584916 500 197 82 221 +81.37 ± 14.54 1 14 107 125 3 +169.27 ± 32.98 ↓
374416 ncm-dbt-04 555386 500 190 67 243 +87.26 ± 13.8 0 12 105 131 2 +183.51 ± 33.24 ↓
374415 ncm-dbt-02 589497 500 195 79 226 +82.1 ± 14.42 0 18 99 132 1 +172.78 ± 34.4 ↓
374414 ncm-dbt-03 588089 500 192 73 235 +84.3 ± 12.95 0 8 116 125 1 +178.11 ± 31.15 ↓
374413 ncm-dbt-01 582778 500 187 74 239 +79.9 ± 14.07 0 16 106 127 1 +167.53 ± 33.17 ↓
374412 ncm-dbt-03 587877 500 205 76 219 +91.71 ± 12.93 0 7 108 134 1 +196.45 ± 32.45 ↓
374411 ncm-dbt-02 590696 500 187 80 233 +75.52 ± 14.23 0 18 108 123 1 +157.24 ± 32.87 ↓
374410 ncm-dbt-04 557061 500 188 69 243 +84.3 ± 13.58 0 11 111 126 2 +176.33 ± 32.16 ↓
374409 ncm-dbt-05 581319 500 189 70 241 +84.3 ± 13.88 0 14 104 131 1 +178.11 ± 33.48 ↓
374408 ncm-dbt-03 589027 500 184 60 256 +88.0 ± 14.41 1 13 99 135 2 +187.16 ± 34.38 ↓
374407 ncm-dbt-02 587155 500 188 62 250 +89.48 ± 14.15 0 13 101 133 3 +187.16 ± 33.99 ↓
374406 ncm-dbt-01 583237 500 189 78 233 +78.44 ± 13.11 0 11 117 122 0 +165.8 ± 31.17 ↓
374405 ncm-dbt-04 558554 500 197 74 229 +87.26 ± 14.1 1 11 104 132 2 +185.33 ± 33.42 ↓
374404 ncm-dbt-05 581236 500 173 75 252 +68.99 ± 14.42 0 21 111 117 1 +142.26 ± 32.42 ↓
374403 ncm-dbt-02 589753 500 184 76 240 +76.25 ± 14.1 1 15 109 125 0 +162.35 ± 32.67 ↓
374402 ncm-dbt-03 588217 500 185 79 236 +74.79 ± 14.63 0 21 103 125 1 +155.54 ± 33.71 ↓
374401 ncm-dbt-04 559091 500 184 81 235 +72.61 ± 14.83 1 20 105 123 1 +152.18 ± 33.38 ↓
374400 ncm-dbt-01 585590 500 185 57 258 +90.96 ± 13.4 0 9 106 133 2 +192.71 ± 32.93 ↓
374399 ncm-dbt-05 583656 500 182 83 235 +69.71 ± 14.03 0 17 119 112 2 +142.26 ± 31.1 ↓
374398 ncm-dbt-03 587113 500 199 59 242 +99.95 ± 13.19 0 9 92 149 0 +219.87 ± 35.65 ↓
374397 ncm-dbt-02 585000 500 179 66 255 +79.9 ± 13.16 0 11 115 124 0 +169.27 ± 31.5 ↓
374396 ncm-dbt-05 582987 500 180 82 238 +68.99 ± 13.56 0 16 120 114 0 +143.89 ± 30.92 ↓
374395 ncm-dbt-01 583782 500 198 73 229 +88.74 ± 14.43 0 16 95 137 2 +187.16 ± 35.14 ↓
374394 ncm-dbt-04 558976 500 188 74 238 +80.63 ± 13.79 0 15 106 129 0 +171.02 ± 33.15 ↓

Commit

Commit ID b209f14b1ee0cda8cbd7fa3a8349e65701d1869f
Author Linmiao Xu
Date 2024-07-09 16:49:28 UTC
Update default main net to nn-e8bac1c07a5a.nnue Created by modifying L2 weights from the previous main net (nn-74f1d263ae9a.nnue) with params found by spsa around 9k / 120k games at 120+1.2. 370 spsa params - L2 weights in nn-74f1d263ae9a.nnue where |val| >= 50 A: 6000, alpha: 0.602, gamma: 0.101 weights: [-127, 127], c_end = 6 To print the spsa params with nnue-pytorch: ``` import features from serialize import NNUEReader feature_set = features.get_feature_set_from_name("HalfKAv2_hm") with open("nn-74f1d263ae9a.nnue", "rb") as f: model = NNUEReader(f, feature_set).model c_end = 6 for i in range(8): for j in range(32): for k in range(30): value = int(model.layer_stacks.l2.weight[32 * i + j, k] * 64) if abs(value) >= 50: print(f"twoW[{i}][{j}][{k}],{value},-127,127,{c_end},0.0020") ``` Among the 370 params, 229 weights were changed. avg change: 0.0961 ± 1.67 range: [-4, 3] The number of weights changed, grouped by layer stack index, shows more weights were modified in the lower piece count buckets: [54, 52, 29, 23, 22, 18, 14, 17] Found with the same method described in: https://github.com/official-stockfish/Stockfish/pull/5459 Passed STC: https://tests.stockfishchess.org/tests/view/668aec9a58083e5fd88239e7 LLR: 3.00 (-2.94,2.94) <0.00,2.00> Total: 52384 W: 13569 L: 13226 D: 25589 Ptnml(0-2): 127, 6141, 13335, 6440, 149 Passed LTC: https://tests.stockfishchess.org/tests/view/668af50658083e5fd8823a0b LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 46974 W: 12006 L: 11668 D: 23300 Ptnml(0-2): 25, 4992, 13121, 5318, 31 closes https://github.com/official-stockfish/Stockfish/pull/5466 bench 1300471
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version