Dev Builds » 20220709-0858

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 10:09:56 1290223 3368 1332 381 1655 +100.84 ± 5.6 0 87 590 976 31 +213.02 ± 14.03
ncm-dbt-02 10:03:36 1286721 3312 1272 361 1679 +98.09 ± 5.49 1 71 626 932 26 +207.47 ± 13.6
ncm-dbt-03 10:10:18 1313527 3376 1302 399 1675 +95.25 ± 5.6 0 91 634 932 31 +198.62 ± 13.53
ncm-dbt-04 10:09:03 1300217 3374 1347 380 1647 +102.45 ± 5.68 0 81 604 956 46 +212.83 ± 13.87
ncm-dbt-05 10:06:35 1303065 3346 1323 394 1629 +99.06 ± 5.64 1 81 616 938 37 +206.86 ± 13.73
ncm-dbt-06 09:42:03 1275090 3224 1305 391 1528 +101.27 ± 5.78 2 70 595 902 43 +210.67 ± 13.96
20000 7881 2306 9813 +99.48 ± 2.3 4 481 3665 5636 214 +208.19 ± 5.62

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
203546 ncm-dbt-02 1193579 144 60 18 66 +104.36 ± 27.18 0 3 26 41 2 +217.63 ± 68.37
203545 ncm-dbt-05 1227408 146 47 19 80 +67.47 ± 25.33 0 5 35 33 0 +140.44 ± 57.87
203544 ncm-dbt-01 1216477 174 70 17 87 +109.29 ± 23.96 0 4 27 55 1 +239.58 ± 67.19
203543 ncm-dbt-03 1224549 176 66 20 90 +92.96 ± 24.25 0 5 33 49 1 +196.15 ± 60.33
203542 ncm-dbt-06 1207873 170 57 13 100 +92.01 ± 24.04 0 4 34 46 1 +193.58 ± 59.23
203541 ncm-dbt-04 1240718 190 78 23 89 +103.53 ± 21.87 0 2 38 53 2 +218.8 ± 55.4
203540 ncm-dbt-02 1196519 500 180 60 260 +85.04 ± 14.92 1 17 95 135 2 +179.9 ± 35.13
203539 ncm-dbt-05 1225666 500 195 59 246 +96.94 ± 14.42 0 13 92 141 4 +204.07 ± 35.72
203538 ncm-dbt-01 1174125 500 200 66 234 +95.44 ± 15.13 0 20 78 150 2 +204.07 ± 38.49
203537 ncm-dbt-03 1231554 500 188 59 253 +91.71 ± 14.34 0 13 99 134 4 +190.85 ± 34.36
203536 ncm-dbt-06 1227339 500 200 61 239 +99.2 ± 14.3 0 8 103 131 8 +202.15 ± 33.42
203535 ncm-dbt-04 1224683 500 213 63 224 +107.54 ± 15.13 0 12 85 144 9 +221.9 ± 37.21
203534 ncm-dbt-05 1220040 500 199 51 250 +106.01 ± 14.67 0 13 81 151 5 +226.0 ± 38.11
203533 ncm-dbt-02 1195142 500 193 46 261 +105.25 ± 14.35 0 12 83 151 4 +226.0 ± 37.66
203532 ncm-dbt-06 1224352 500 201 70 229 +93.2 ± 16.19 2 17 86 138 7 +192.71 ± 36.85
203531 ncm-dbt-01 1212634 500 208 53 239 +111.37 ± 14.06 0 7 88 148 7 +236.51 ± 36.44
203530 ncm-dbt-04 1214668 500 181 55 264 +89.48 ± 15.44 0 18 95 130 7 +179.9 ± 35.13
203529 ncm-dbt-03 1253555 500 197 59 244 +98.44 ± 13.34 0 10 92 148 0 +215.85 ± 35.68
203528 ncm-dbt-05 1219085 500 200 63 237 +97.69 ± 15.31 0 15 91 136 8 +198.34 ± 35.92
203527 ncm-dbt-04 1233645 500 200 51 249 +106.77 ± 14.05 0 10 85 151 4 +230.16 ± 37.2
203526 ncm-dbt-06 1225860 500 204 70 226 +95.44 ± 14.99 0 14 95 134 7 +194.57 ± 35.13
203525 ncm-dbt-03 1244954 500 210 65 225 +103.73 ± 14.8 0 12 88 143 7 +215.85 ± 36.55
203524 ncm-dbt-02 1194403 500 191 60 249 +93.2 ± 12.95 0 7 106 136 1 +200.24 ± 32.81
203523 ncm-dbt-01 1208252 500 204 58 238 +104.49 ± 14.35 0 11 87 147 5 +221.9 ± 36.76
203522 ncm-dbt-05 1219647 500 203 63 234 +99.95 ± 14.31 0 7 105 129 9 +202.15 ± 32.99
203521 ncm-dbt-04 1229123 500 197 48 255 +106.77 ± 14.67 0 11 86 146 7 +223.94 ± 36.98
203520 ncm-dbt-02 1195390 500 198 54 248 +102.97 ± 14.64 0 9 97 135 9 +209.91 ± 34.63
203519 ncm-dbt-01 1212303 500 198 53 249 +103.73 ± 14.95 0 16 77 153 4 +221.9 ± 38.94
203518 ncm-dbt-06 1237968 500 213 60 227 +109.83 ± 14.99 0 10 87 143 10 +226.0 ± 36.75
203517 ncm-dbt-03 1246348 500 187 59 254 +90.96 ± 14.47 0 15 95 137 3 +190.85 ± 35.14
203516 ncm-dbt-05 1232492 500 198 44 258 +110.6 ± 14.06 1 9 77 161 2 +247.41 ± 39.15
203515 ncm-dbt-04 1216458 500 215 57 228 +113.68 ± 14.69 0 10 80 152 8 +240.82 ± 38.39
203514 ncm-dbt-02 1194496 500 191 59 250 +93.95 ± 13.92 0 11 99 137 3 +198.34 ± 34.31
203513 ncm-dbt-01 1201540 500 185 66 249 +84.3 ± 14.9 0 17 102 126 5 +171.02 ± 33.87
203512 ncm-dbt-03 1253835 500 200 48 252 +109.07 ± 14.99 0 12 82 148 8 +228.08 ± 37.89
203511 ncm-dbt-06 1242632 500 206 49 245 +112.91 ± 13.89 0 7 85 152 6 +243.0 ± 37.13
203510 ncm-dbt-04 1245576 500 188 62 250 +89.48 ± 15.44 0 17 98 127 8 +178.11 ± 34.58
203509 ncm-dbt-05 1220063 500 204 72 224 +93.95 ± 15.11 0 16 92 136 6 +192.71 ± 35.72
203508 ncm-dbt-02 1195427 500 198 49 253 +106.77 ± 13.39 0 8 87 153 2 +234.38 ± 36.71
203507 ncm-dbt-03 1218024 500 184 67 249 +82.83 ± 15.41 0 19 102 122 7 +164.07 ± 33.88
203506 ncm-dbt-01 1209878 500 194 51 255 +102.22 ± 13.86 0 7 99 138 6 +213.85 ± 34.13
203505 ncm-dbt-06 1228471 500 202 57 241 +103.73 ± 13.71 0 8 93 145 4 +221.9 ± 35.4
175826 ncm-dbt-02 1449111 18 6 4 8 +38.76 ± 73.41 0 1 5 3 0 +78.52 ± 162.75
175825 ncm-dbt-04 1423879 34 14 5 15 +94.18 ± 44.67 0 0 8 9 0 +204.71 ± 123.58
175824 ncm-dbt-01 1438907 44 18 4 22 +114.51 ± 46.23 0 1 6 15 0 +261.29 ± 160.77
175823 ncm-dbt-05 1444302 50 17 7 26 +70.42 ± 49.53 0 2 12 10 1 +130.94 ± 101.68
175822 ncm-dbt-03 1445738 50 17 5 28 +85.01 ± 41.73 0 0 14 10 1 +164.04 ± 86.61
175821 ncm-dbt-02 1445030 50 19 2 29 +122.99 ± 42.31 0 0 9 15 1 +263.38 ± 120.78
175820 ncm-dbt-04 1422604 50 22 5 23 +123.0 ± 36.12 0 0 8 17 0 +288.02 ± 131.84
175819 ncm-dbt-01 1439925 50 19 6 25 +92.45 ± 47.1 0 2 8 15 0 +200.24 ± 129.69
175818 ncm-dbt-05 1446036 50 19 4 27 +107.53 ± 42.62 0 1 8 16 0 +240.82 ± 132.75
175817 ncm-dbt-03 1447056 50 14 4 32 +70.43 ± 45.16 0 2 11 12 0 +147.19 ± 107.43
175816 ncm-dbt-02 1446614 50 20 5 25 +107.52 ± 56.64 0 2 8 13 2 +200.24 ± 129.69
175815 ncm-dbt-04 1421760 50 20 4 26 +115.2 ± 36.62 0 0 9 16 0 +263.38 ± 120.78
175814 ncm-dbt-01 1437174 50 18 4 28 +99.93 ± 36.88 0 0 11 14 0 +219.84 ± 104.3
175813 ncm-dbt-05 1440357 50 21 5 24 +115.19 ± 42.56 0 0 10 14 1 +240.79 ± 111.83
175812 ncm-dbt-03 1442683 50 20 5 25 +107.53 ± 42.62 0 1 8 16 0 +240.82 ± 132.75
175811 ncm-dbt-06 1437764 4 2 1 1 +88.51 ± 137.63 0 0 1 1 0 +190.5 ± 634.21
175810 ncm-dbt-02 1448225 50 16 4 30 +85.04 ± 41.74 0 1 11 13 0 +181.7 ± 107.11
175809 ncm-dbt-01 1441242 50 18 3 29 +107.52 ± 52.35 0 2 7 15 1 +219.87 ± 140.03
175808 ncm-dbt-04 1429279 50 19 7 24 +85.02 ± 46.58 0 1 12 11 1 +164.07 ± 100.82
175807 ncm-dbt-05 1438622 50 20 7 23 +92.43 ± 42.19 0 0 13 11 1 +181.68 ± 91.94
175806 ncm-dbt-06 1443557 50 20 10 20 +70.43 ± 45.16 0 2 11 12 0 +147.19 ± 107.43
175805 ncm-dbt-03 1440503 50 19 8 23 +77.7 ± 45.94 0 2 10 13 0 +164.07 ± 113.84

Commit

Commit ID 2e02dd79366e6f17df5b0599048937289fd5819e
Author Joost VandeVondele
Date 2022-07-09 08:58:04 UTC
Limit the researching at same depth. If the elapsed time is close to the available time, the time management thread can signal that the next iterations should be searched at the same depth (Threads.increaseDepth = false). While the rootDepth increases, the adjustedDepth is kept constant with the searchAgainCounter. In exceptional cases, when threading is used and the master thread, which controls the time management, signals to not increaseDepth, but by itself takes a long time to finish the iteration, the helper threads can search repeatedly at the same depth. This search finishes more and more quickly, leading to helper threads that report a rootDepth of MAX_DEPTH (245). The latter is not optimal as it is confusing for the user, stops search on these threads, and leads to an incorrect bias in the thread voting scheme. Probably with only a small impact on strength. This behavior was observed almost two years ago, see https://github.com/official-stockfish/Stockfish/issues/2717 This patch fixes #2717 by ensuring the effective depth increases at once every four iterations, even in increaseDepth is false. Depth 245 searches (for non-trivial positions) were indeed absent with this patch, but frequent with master in the tests below: https://discord.com/channels/435943710472011776/813919248455827515/994872720800088095 Total pgns: 2173 Base: 2867 Patch: 0 it passed non-regression testing in various setups: SMP STC: https://tests.stockfishchess.org/tests/view/62bfecc96178ffe6394ba036 LLR: 2.94 (-2.94,2.94) <-2.25,0.25> Total: 37288 W: 10171 L: 10029 D: 17088 Ptnml(0-2): 75, 3777, 10793, 3929, 70 SMP LTC: https://tests.stockfishchess.org/tests/view/62c08f6f49b62510394be066 LLR: 2.94 (-2.94,2.94) <-2.25,0.25> Total: 190568 W: 52125 L: 52186 D: 86257 Ptnml(0-2): 70, 17854, 59504, 17779, 77 LTC: https://tests.stockfishchess.org/tests/view/62c08b6049b62510394bdfb6 LLR: 2.96 (-2.94,2.94) <-2.25,0.25> Total: 48120 W: 13204 L: 13083 D: 21833 Ptnml(0-2): 54, 4458, 14919, 4571, 58 Special thanks to miguel-I, Disservin, ruicoelhopedro and others for analysing the problem, the data, and coming up with the key insight, needed to fix this longstanding issue. closes https://github.com/official-stockfish/Stockfish/pull/4104 Bench: 5182295
Copyright 2011–2024 Next Chess Move LLC