Dev Builds » 20230711-2056

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:52:47 584158 4000 1390 697 1913 +60.81 ± 5.1 2 178 957 851 12 +123.61 ± 10.97
ncm-dbt-02 06:52:30 585804 4000 1354 678 1968 +59.29 ± 5.17 3 195 932 863 7 +121.46 ± 11.14
ncm-dbt-03 06:52:57 584734 4000 1375 660 1965 +62.78 ± 5.08 1 171 953 862 13 +127.56 ± 10.99
ncm-dbt-04 06:52:53 568564 4012 1368 722 1922 +56.43 ± 5.07 2 194 970 836 4 +115.63 ± 10.9
ncm-dbt-05 06:53:20 577364 3988 1345 671 1972 +59.29 ± 5.08 2 180 963 840 9 +120.87 ± 10.93
20000 6832 3428 9740 +59.71 ± 2.28 10 918 4775 4252 45 +121.81 ± 4.91

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
426113 ncm-dbt-04 568037 12 3 3 6 -0.0 ± 81.87 0 1 4 1 0 -0.0 ± 173.65 ↓
426112 ncm-dbt-05 577110 488 164 84 240 +57.47 ± 14.19 0 22 120 102 0 +118.28 ± 31.03 ↓
426111 ncm-dbt-01 585337 500 179 80 241 +69.71 ± 14.45 0 19 116 112 3 +140.62 ± 31.61 ↓
426110 ncm-dbt-03 585042 500 168 83 249 +59.64 ± 14.32 0 23 120 106 1 +121.46 ± 31.09 ↓
426109 ncm-dbt-02 587282 500 165 74 261 +63.94 ± 14.23 0 20 121 107 2 +129.35 ± 30.88 ↓
426108 ncm-dbt-04 567720 500 167 88 245 +55.36 ± 15.19 0 30 113 105 2 +110.6 ± 32.15 ↓
426107 ncm-dbt-05 578424 500 178 79 243 +69.71 ± 14.31 0 21 109 120 0 +145.54 ± 32.74 ↓
426106 ncm-dbt-03 585506 500 182 86 232 +67.54 ± 13.51 0 15 125 109 1 +138.99 ± 30.12 ↓
426105 ncm-dbt-01 584327 500 170 94 236 +53.22 ± 13.15 0 17 141 91 1 +107.54 ± 27.91 ↓
426104 ncm-dbt-02 584076 500 175 89 236 +60.36 ± 14.76 0 26 113 110 1 +123.02 ± 32.15 ↓
426103 ncm-dbt-04 568037 500 178 98 224 +56.07 ± 14.01 0 22 127 100 1 +113.68 ± 30.06 ↓
426102 ncm-dbt-05 581319 500 167 94 239 +51.09 ± 14.18 0 26 125 99 0 +104.49 ± 30.42 ↓
426101 ncm-dbt-01 585253 500 180 95 225 +59.64 ± 14.18 0 22 122 105 1 +121.46 ± 30.78 ↓
426100 ncm-dbt-03 586308 500 176 85 239 +63.95 ± 13.8 0 19 121 110 0 +132.54 ± 30.86 ↓
426099 ncm-dbt-02 587113 500 170 89 241 +56.78 ± 15.25 0 31 108 110 1 +115.23 ± 32.87 ↓
426098 ncm-dbt-04 568156 500 165 77 258 +61.79 ± 14.95 1 24 112 112 1 +127.76 ± 32.29 ↓
426097 ncm-dbt-05 572477 500 166 74 260 +64.66 ± 14.8 1 21 115 111 2 +132.54 ± 31.82 ↓
426096 ncm-dbt-02 587028 500 167 89 244 +54.65 ± 14.08 0 24 124 102 0 +112.14 ± 30.53 ↓
426095 ncm-dbt-03 582652 500 163 87 250 +53.22 ± 14.56 0 26 124 98 2 +106.01 ± 30.56 ↓
426094 ncm-dbt-01 582694 500 170 85 245 +59.64 ± 14.99 1 23 119 104 3 +119.89 ± 31.25 ↓
426093 ncm-dbt-04 569789 500 165 85 250 +56.07 ± 14.56 0 27 116 107 0 +115.23 ± 31.71 ↓
426092 ncm-dbt-01 584243 500 175 101 224 +51.8 ± 14.75 0 29 119 101 1 +104.49 ± 31.29 ↓
426091 ncm-dbt-03 582861 500 169 83 248 +60.36 ± 14.35 0 23 119 107 1 +123.02 ± 31.23 ↓
426090 ncm-dbt-05 581860 500 155 88 257 +46.84 ± 15.27 0 35 114 100 1 +93.95 ± 31.99 ↓
426089 ncm-dbt-02 586477 500 166 84 250 +57.5 ± 14.89 1 25 116 107 1 +118.33 ± 31.7 ↓
426088 ncm-dbt-04 567442 500 175 89 236 +60.36 ± 13.5 0 18 128 104 0 +124.6 ± 29.8 ↓
426087 ncm-dbt-05 578753 500 173 79 248 +66.1 ± 14.31 0 18 124 104 4 +130.94 ± 30.38 ↓
426086 ncm-dbt-01 582277 500 188 74 238 +80.63 ± 13.94 0 14 110 124 2 +167.53 ± 32.45 ↓
426085 ncm-dbt-03 584916 500 169 87 244 +57.5 ± 14.76 1 24 118 106 1 +118.33 ± 31.4 ↓
426084 ncm-dbt-02 586139 500 166 69 265 +68.27 ± 14.4 0 21 112 116 1 +140.62 ± 32.26 ↓
426083 ncm-dbt-04 568792 500 176 87 237 +62.51 ± 14.17 0 22 117 111 0 +129.35 ± 31.52 ↓
426082 ncm-dbt-05 572800 500 170 84 246 +60.36 ± 13.35 0 17 130 103 0 +124.6 ± 29.47 ↓
426081 ncm-dbt-04 568116 500 173 94 233 +55.36 ± 14.53 0 27 117 106 0 +113.68 ± 31.56 ↓
426080 ncm-dbt-02 583405 500 172 101 227 +49.67 ± 14.9 1 28 121 99 1 +101.46 ± 31.01 ↓
426079 ncm-dbt-01 582110 500 164 95 241 +48.25 ± 15.47 1 33 113 102 1 +98.44 ± 32.13 ↓
426078 ncm-dbt-03 584453 500 176 68 256 +76.25 ± 13.51 0 13 117 119 1 +158.93 ± 31.27 ↓
426077 ncm-dbt-01 587028 500 164 73 263 +63.95 ± 14.09 0 21 117 112 0 +132.54 ± 31.5 ↓
426076 ncm-dbt-05 576169 500 172 89 239 +58.21 ± 14.39 1 20 126 101 2 +118.33 ± 30.18 ↓
426075 ncm-dbt-02 584916 500 173 83 244 +63.23 ± 14.34 1 20 117 112 0 +132.54 ± 31.5 ↓
426074 ncm-dbt-04 570989 500 166 101 233 +45.42 ± 13.87 1 23 136 90 0 +93.95 ± 28.85 ↓
426073 ncm-dbt-03 586139 500 172 81 247 +63.94 ± 15.82 0 28 109 107 6 +123.02 ± 32.74 ↓

Commit

Commit ID af110e02ec96cdb46cf84c68252a1da15a902395
Author Joost VandeVondele
Date 2023-07-11 20:56:49 UTC
Remove classical evaluation since the introduction of NNUE (first released with Stockfish 12), we have maintained the classical evaluation as part of SF in frozen form. The idea that this code could lead to further inputs to the NN or search did not materialize. Now, after five releases, this PR removes the classical evaluation from SF. Even though this evaluation is probably the best of its class, it has become unimportant for the engine's strength, and there is little need to maintain this code (roughly 25% of SF) going forward, or to expend resources on trying to improve its integration in the NNUE eval. Indeed, it had still a very limited use in the current SF, namely for the evaluation of positions that are nearly decided based on material difference, where the speed of the classical evaluation outweights its inaccuracies. This impact on strength is small, roughly 2Elo, and probably decreasing in importance as the TC grows. Potentially, removal of this code could lead to the development of techniques to have faster, but less accurate NN evaluation, for certain positions. STC https://tests.stockfishchess.org/tests/view/64a320173ee09aa549c52157 Elo: -2.35 ± 1.1 (95%) LOS: 0.0% Total: 100000 W: 24916 L: 25592 D: 49492 Ptnml(0-2): 287, 12123, 25841, 11477, 272 nElo: -4.62 ± 2.2 (95%) PairsRatio: 0.95 LTC https://tests.stockfishchess.org/tests/view/64a320293ee09aa549c5215b Elo: -1.74 ± 1.0 (95%) LOS: 0.0% Total: 100000 W: 25010 L: 25512 D: 49478 Ptnml(0-2): 44, 11069, 28270, 10579, 38 nElo: -3.72 ± 2.2 (95%) PairsRatio: 0.96 VLTC SMP https://tests.stockfishchess.org/tests/view/64a3207c3ee09aa549c52168 Elo: -1.70 ± 0.9 (95%) LOS: 0.0% Total: 100000 W: 25673 L: 26162 D: 48165 Ptnml(0-2): 8, 9455, 31569, 8954, 14 nElo: -3.95 ± 2.2 (95%) PairsRatio: 0.95 closes https://github.com/official-stockfish/Stockfish/pull/4674 Bench: 1444646
Copyright 2011–2025 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version