Dev Builds » 20240709-1635

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:53:13 584620 3990 1540 580 1870 +85.27 ± 4.99 2 108 830 1038 17 +178.86 ± 11.79
ncm-dbt-02 06:53:27 588085 4000 1506 623 1871 +77.98 ± 5.01 3 121 881 980 15 +162.14 ± 11.43
ncm-dbt-03 06:54:25 587127 4016 1512 566 1938 +83.41 ± 4.98 1 120 830 1046 11 +175.5 ± 11.81
ncm-dbt-04 06:54:23 557378 4000 1509 579 1912 +82.28 ± 4.85 0 105 870 1015 10 +172.78 ± 11.49
ncm-dbt-05 06:54:52 581972 3994 1514 569 1911 +83.79 ± 5.05 2 113 843 1016 23 +173.97 ± 11.7
20000 7581 2917 9502 +82.54 ± 2.23 8 567 4254 5095 76 +172.61 ± 5.2

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
374781 ncm-dbt-03 586901 16 5 6 5 -21.71 ± 40.35 0 1 7 0 0 -43.61 ± 82.52 ↓
374780 ncm-dbt-01 585928 490 183 72 235 +80.09 ± 13.49 0 12 110 123 0 +169.74 ± 32.34 ↓
374779 ncm-dbt-05 579537 494 187 71 236 +83.13 ± 13.97 0 11 113 119 4 +169.9 ± 31.8 ↓
374778 ncm-dbt-02 588515 500 186 85 229 +71.16 ± 13.64 1 12 123 113 1 +148.85 ± 30.34 ↓
374777 ncm-dbt-04 557672 500 184 61 255 +87.26 ± 13.64 0 11 107 130 2 +183.51 ± 32.85 ↓
374776 ncm-dbt-03 589240 500 198 71 231 +90.22 ± 13.86 0 11 104 132 3 +189.0 ± 33.39 ↓
374775 ncm-dbt-01 584453 500 193 82 225 +78.43 ± 13.42 0 11 119 118 2 +162.35 ± 30.85 ↓
374774 ncm-dbt-05 582945 500 201 78 221 +87.26 ± 14.25 1 11 105 130 3 +183.51 ± 33.24 ↓
374773 ncm-dbt-04 556985 500 182 74 244 +76.25 ± 14.67 0 21 101 127 1 +158.93 ± 34.05 ↓
374772 ncm-dbt-02 588515 500 198 85 217 +79.9 ± 14.21 1 12 113 121 3 +165.8 ± 31.92 ↓
374771 ncm-dbt-03 585000 500 190 70 240 +85.04 ± 14.49 0 16 101 130 3 +176.33 ± 34.04 ↓
374770 ncm-dbt-01 584411 500 189 74 237 +81.36 ± 13.81 0 13 111 124 2 +169.27 ± 32.25 ↓
374769 ncm-dbt-04 559438 500 193 73 234 +85.04 ± 12.8 0 8 114 128 0 +181.7 ± 31.48 ↓
374768 ncm-dbt-02 588004 500 178 72 250 +74.79 ± 14.63 0 20 106 122 2 +153.86 ± 33.21 ↓
374767 ncm-dbt-05 582318 500 195 72 233 +87.26 ± 14.1 1 11 104 132 2 +185.33 ± 33.42 ↓
374766 ncm-dbt-03 587664 500 186 62 252 +88.0 ± 13.66 0 12 103 134 1 +187.16 ± 33.6 ↓
374765 ncm-dbt-01 582652 500 189 68 243 +85.78 ± 14.65 0 16 101 129 4 +176.33 ± 34.04 ↓
374764 ncm-dbt-05 580904 500 183 66 251 +82.83 ± 14.58 0 16 105 125 4 +169.27 ± 33.34 ↓
374763 ncm-dbt-04 557940 500 193 58 249 +96.19 ± 13.48 0 9 99 140 2 +206.01 ± 34.24 ↓
374762 ncm-dbt-02 587494 500 192 76 232 +82.1 ± 13.98 1 12 108 128 1 +174.55 ± 32.75 ↓
374761 ncm-dbt-03 587707 500 195 70 235 +88.74 ± 13.83 1 10 103 135 1 +190.85 ± 33.57 ↓
374760 ncm-dbt-01 583112 500 200 71 229 +91.71 ± 14.49 0 13 100 132 5 +189.0 ± 34.18 ↓
374759 ncm-dbt-04 557061 500 195 83 222 +79.17 ± 13.6 0 13 113 123 1 +165.8 ± 31.92 ↓
374758 ncm-dbt-05 580779 500 188 77 235 +78.43 ± 14.17 0 16 109 123 2 +162.35 ± 32.67 ↓
374757 ncm-dbt-02 587877 500 191 78 231 +79.9 ± 13.92 0 14 111 123 2 +165.8 ± 32.28 ↓
374756 ncm-dbt-03 588047 500 194 75 231 +84.3 ± 14.03 0 15 102 132 1 +178.11 ± 33.85 ↓
374755 ncm-dbt-01 587622 500 194 72 234 +86.52 ± 14.67 2 11 103 131 3 +183.51 ± 33.63 ↓
374754 ncm-dbt-04 555956 500 187 77 236 +77.7 ± 14.15 0 17 107 125 1 +162.35 ± 33.02 ↓
374753 ncm-dbt-05 583405 500 194 69 237 +88.74 ± 14.13 0 14 99 135 2 +187.16 ± 34.38 ↓
374752 ncm-dbt-03 583447 500 175 87 238 +61.79 ± 14.41 0 23 117 109 1 +126.18 ± 31.53 ↓
374751 ncm-dbt-02 589027 500 200 72 228 +90.96 ± 13.72 0 10 105 132 3 +190.85 ± 33.16 ↓
374750 ncm-dbt-01 582903 500 187 71 242 +82.1 ± 14.56 0 20 94 136 0 +174.55 ± 35.29 ↓
374749 ncm-dbt-05 582485 500 182 64 254 +83.57 ± 14.16 0 15 104 129 2 +174.55 ± 33.5 ↓
374748 ncm-dbt-03 588771 500 181 72 247 +76.97 ± 14.7 0 21 100 128 1 +160.64 ± 34.22 ↓
374747 ncm-dbt-04 555994 500 192 84 224 +76.25 ± 13.36 0 11 122 115 2 +157.24 ± 30.38 ↓
374746 ncm-dbt-02 586520 500 181 73 246 +76.25 ± 14.95 0 22 100 126 2 +157.24 ± 34.21 ↓
374745 ncm-dbt-01 585885 500 205 70 225 +96.19 ± 13.8 0 12 92 145 1 +207.95 ± 35.71 ↓
374744 ncm-dbt-02 588728 500 180 82 238 +68.99 ± 14.14 0 19 115 115 1 +142.26 ± 31.77 ↓
374743 ncm-dbt-04 557978 500 183 69 248 +80.63 ± 13.94 0 15 107 127 1 +169.27 ± 32.98 ↓
374742 ncm-dbt-03 587367 500 188 53 259 +96.19 ± 13.48 0 11 93 146 0 +209.91 ± 35.49 ↓
374741 ncm-dbt-05 583405 500 184 72 244 +79.17 ± 14.9 0 19 104 123 4 +160.64 ± 33.54 ↓

Commit

Commit ID 5752529cabb3270e055147709ff0847e4d59ec22
Author Linmiao Xu
Date 2024-07-09 16:35:23 UTC
Update default main net to nn-74f1d263ae9a.nnue Created by setting output weights (256) and biases (8) of the previous main net nn-ddcfb9224cdb.nnue to values found around 12k / 120k spsa games at 120+1.2 This used modified fishtest dev workers to construct .nnue files from spsa params, then load them with EvalFile when running tests: https://github.com/linrock/fishtest/tree/spsa-file-modified-nnue/worker Inspired by researching loading spsa params from files: https://github.com/official-stockfish/fishtest/pull/1926 Scripts for modifying nnue files and preparing params: https://github.com/linrock/nnue-pytorch/tree/no-gpu-modify-nnue spsa params: weights: [-127, 127], c_end = 6 biases: [-8192, 8192], c_end = 64 Example of reading output weights and biases from the previous main net using nnue-pytorch and printing spsa params in a format compatible with fishtest: ``` import features from serialize import NNUEReader feature_set = features.get_feature_set_from_name("HalfKAv2_hm") with open("nn-ddcfb9224cdb.nnue", "rb") as f: model = NNUEReader(f, feature_set).model c_end_weights = 6 c_end_biases = 64 for i in range(8): for j in range(32): value = round(int(model.layer_stacks.output.weight[i, j] * 600 * 16) / 127) print(f"oW[{i}][{j}],{value},-127,127,{c_end_weights},0.0020") for i in range(8): value = int(model.layer_stacks.output.bias[i] * 600 * 16) print(f"oB[{i}],{value},-8192,8192,{c_end_biases},0.0020") ``` For more info on spsa tuning params in nets: https://github.com/official-stockfish/Stockfish/pull/5149 https://github.com/official-stockfish/Stockfish/pull/5254 Passed STC: https://tests.stockfishchess.org/tests/view/66894d64e59d990b103f8a37 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 32000 W: 8443 L: 8137 D: 15420 Ptnml(0-2): 80, 3627, 8309, 3875, 109 Passed LTC: https://tests.stockfishchess.org/tests/view/6689668ce59d990b103f8b8b LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 172176 W: 43822 L: 43225 D: 85129 Ptnml(0-2): 97, 18821, 47633, 19462, 75 closes https://github.com/official-stockfish/Stockfish/pull/5459 bench 1120091
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version