Dev Builds » 20220917-0713

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:58:31 1246417 3368 1340 388 1640 +100.95 ± 5.52 0 85 585 991 23 +215.68 ± 14.09
ncm-dbt-02 10:00:22 1258188 3364 1349 372 1643 +103.89 ± 5.4 1 66 594 997 24 +223.53 ± 13.97
ncm-dbt-03 09:59:10 1261441 3370 1316 360 1694 +101.34 ± 5.47 0 72 616 966 31 +214.33 ± 13.72
ncm-dbt-04 10:01:06 1264422 3368 1344 378 1646 +102.53 ± 5.51 1 71 606 973 33 +217.16 ± 13.83
ncm-dbt-05 09:58:42 1136959 3360 1336 390 1634 +100.53 ± 5.69 1 86 597 958 38 +210.43 ± 13.95
ncm-dbt-06 09:26:13 1232215 3170 1281 385 1504 +100.95 ± 5.65 0 75 562 925 23 +215.24 ± 14.38
20000 7966 2273 9761 +101.71 ± 2.26 3 455 3560 5810 172 +216.05 ± 5.71

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
197642 ncm-dbt-05 1221248 160 60 22 78 +84.12 ± 24.73 0 4 35 40 1 +173.89 ± 58.06
197641 ncm-dbt-02 1227431 160 58 21 81 +81.82 ± 26.23 0 5 35 38 2 +162.99 ± 58.21
197640 ncm-dbt-01 1236176 168 59 12 97 +99.86 ± 25.97 0 5 29 48 2 +207.81 ± 64.55
197639 ncm-dbt-06 1206509 170 72 22 76 +105.29 ± 24.3 0 4 28 52 1 +228.32 ± 65.89
197638 ncm-dbt-04 1224627 170 74 19 77 +116.59 ± 24.32 0 2 29 51 3 +247.28 ± 64.51
197637 ncm-dbt-03 1242975 172 69 24 79 +93.06 ± 23.88 0 3 37 44 2 +190.85 ± 56.21
197636 ncm-dbt-01 1230250 500 190 56 254 +95.44 ± 13.94 0 12 94 142 2 +204.07 ± 35.31
197635 ncm-dbt-02 1215642 500 198 54 248 +102.97 ± 13.87 0 11 86 151 2 +223.94 ± 36.98
197634 ncm-dbt-03 1244748 500 197 48 255 +106.77 ± 13.89 0 9 87 150 4 +230.16 ± 36.73
197633 ncm-dbt-05 1237732 500 204 56 240 +106.01 ± 14.51 1 8 89 146 6 +226.0 ± 36.29
197632 ncm-dbt-04 1250194 500 200 48 252 +109.07 ± 13.23 0 8 83 158 1 +243.0 ± 37.64
197631 ncm-dbt-06 1260489 500 199 57 244 +101.46 ± 13.69 0 8 96 142 4 +215.85 ± 34.78
197630 ncm-dbt-03 1230946 500 189 58 253 +93.2 ± 14.36 0 15 91 142 2 +198.34 ± 35.92
197629 ncm-dbt-01 1175498 500 199 65 236 +95.44 ± 13.94 0 11 97 139 3 +202.15 ± 34.7
197628 ncm-dbt-02 1224524 500 192 45 263 +105.25 ± 14.2 1 8 88 149 4 +228.08 ± 36.51
197627 ncm-dbt-05 1244541 500 206 58 236 +106.01 ± 15.26 0 13 85 143 9 +217.85 ± 37.2
197626 ncm-dbt-06 1223467 500 196 64 240 +93.95 ± 14.82 0 17 87 143 3 +198.34 ± 36.7
197625 ncm-dbt-04 1227136 500 208 58 234 +107.54 ± 14.67 1 9 85 149 6 +230.16 ± 37.2
197456 ncm-dbt-02 1231590 500 195 62 243 +94.69 ± 14.09 0 13 93 142 2 +202.15 ± 35.52
197455 ncm-dbt-05 1225411 500 200 60 240 +99.95 ± 15.05 0 18 77 152 3 +213.85 ± 38.83
197454 ncm-dbt-01 1188581 500 205 52 243 +109.83 ± 14.53 0 13 75 158 4 +238.66 ± 39.57
197453 ncm-dbt-06 1235357 500 205 64 231 +100.7 ± 14.92 0 14 87 143 6 +209.91 ± 36.75
197452 ncm-dbt-04 1250324 500 200 66 234 +95.44 ± 14.25 0 12 96 138 4 +200.24 ± 34.91
197451 ncm-dbt-03 1236003 500 184 44 272 +99.95 ± 13.68 0 11 89 149 1 +217.85 ± 36.33
197450 ncm-dbt-02 1229133 500 203 55 242 +106.01 ± 13.72 0 8 90 148 4 +228.08 ± 36.04
197449 ncm-dbt-05 1211230 500 195 55 250 +99.95 ± 14.46 0 11 94 139 6 +207.95 ± 35.29
197448 ncm-dbt-06 1215132 500 194 64 242 +92.45 ± 13.59 0 11 99 139 1 +198.34 ± 34.31
197447 ncm-dbt-01 1245415 500 194 75 231 +84.3 ± 14.62 0 19 94 136 1 +178.11 ± 35.3
197446 ncm-dbt-03 1200507 500 192 47 261 +103.73 ± 14.5 0 12 86 147 5 +219.87 ± 36.98
197445 ncm-dbt-04 1224328 500 188 51 261 +97.69 ± 14.73 0 13 93 138 6 +202.15 ± 35.52
197444 ncm-dbt-05 1229641 500 202 48 250 +110.6 ± 14.38 0 10 82 152 6 +236.51 ± 37.9
197443 ncm-dbt-04 1230839 500 203 60 237 +102.22 ± 14.33 0 11 90 144 5 +215.85 ± 36.11
197442 ncm-dbt-03 1221315 500 198 63 239 +96.19 ± 13.8 0 8 104 133 5 +200.24 ± 33.24
197441 ncm-dbt-01 1206197 500 208 58 234 +107.54 ± 13.56 0 9 84 155 2 +236.51 ± 37.42
197440 ncm-dbt-02 1213405 500 211 56 233 +111.37 ± 14.06 0 9 82 154 5 +240.82 ± 37.9
197439 ncm-dbt-06 1238748 500 202 47 251 +111.37 ± 14.06 0 10 79 157 4 +243.0 ± 38.64
197438 ncm-dbt-01 1214020 500 205 54 241 +108.3 ± 14.83 0 13 79 152 6 +230.16 ± 38.58
197437 ncm-dbt-02 1235177 500 205 51 244 +110.6 ± 13.39 0 8 82 158 2 +245.2 ± 37.88
197436 ncm-dbt-05 1222715 500 186 64 250 +86.52 ± 14.08 0 13 105 129 3 +179.9 ± 33.27
197435 ncm-dbt-04 1238134 500 198 55 247 +102.22 ± 14.33 0 10 93 141 6 +213.85 ± 35.47
197434 ncm-dbt-06 1245808 500 213 67 220 +104.49 ± 14.19 0 11 86 149 4 +223.94 ± 36.98
197433 ncm-dbt-03 1233531 500 208 54 238 +110.6 ± 14.84 0 10 85 146 9 +230.16 ± 37.2
177272 ncm-dbt-05 503157 200 83 27 90 +99.95 ± 26.35 0 9 30 57 4 +200.24 ± 62.37
177271 ncm-dbt-03 1481508 198 79 22 97 +102.92 ± 23.22 0 4 37 55 3 +212.59 ± 56.81
177270 ncm-dbt-04 1469797 198 73 21 104 +93.43 ± 23.61 0 6 37 54 2 +193.2 ± 56.86
177269 ncm-dbt-02 1488607 204 87 28 89 +103.43 ± 22.75 0 4 38 57 3 +214.36 ± 56.02
177268 ncm-dbt-01 1475206 200 80 16 104 +115.22 ± 22.5 0 3 33 61 3 +246.3 ± 60.46

Commit

Commit ID 154e7afed0fe9c6f45a2aee8ef6f38d44076cb19
Author atumanian
Date 2022-09-17 07:13:07 UTC
Simplify trend and optimism. This patch simplifies the formulas used to compute the trend and optimism values before each search iteration. As a side effect, this removes the parameters which make the relationship between the displayed evaluation value and the expected game result asymmetric. I've also provided links to the results of isotonic regression analysis of the relationship between the evaluation and game result (statistical data and a graph) for both tests, which demonstrate that the new version has a more symmetric relationship: STC: [Data and graph](https://github.com/official-stockfish/Stockfish/discussions/4150#discussioncomment-3548954) LTC: [Data and graph](https://github.com/official-stockfish/Stockfish/discussions/4150#discussioncomment-3626311) See also https://github.com/official-stockfish/Stockfish/issues/4142 passed STC: https://tests.stockfishchess.org/tests/view/6313f44b8202a039920e27e6 LLR: 2.96 (-2.94,2.94) <-1.75,0.25> Total: 108016 W: 28903 L: 28760 D: 50353 Ptnml(0-2): 461, 12075, 28850, 12104, 518 passed LTC: https://tests.stockfishchess.org/tests/view/631de45db85daa436625dfe6 LLR: 3.01 (-2.94,2.94) <-1.75,0.25> Total: 34792 W: 9412 L: 9209 D: 16171 Ptnml(0-2): 24, 3374, 10397, 3577, 24 Furthermore, this does not measurably impact Elo strength against weaker engines, as demonstrated in a match of master and patch vs SF13: This patch vs SF 13: https://tests.stockfishchess.org/tests/view/631fa34ae1612778c344c6eb Elo: 141.66 +-1.2 (95%) LOS: 100.0% Total: 100000 W: 48182 L: 9528 D: 42290 Ptnml(0-2): 96, 1426, 13277, 30130, 5071 nElo: 284.13 +-3.3 (95%) PairsRatio: 23.13 Master vs SF 13: https://tests.stockfishchess.org/tests/view/631fa3ece1612778c344c6ff Elo: 143.26 +-1.2 (95%) LOS: 100.0% Total: 100000 W: 48525 L: 9479 D: 41996 Ptnml(0-2): 94, 1537, 13098, 29771, 5500 nElo: 281.70 +-3.3 (95%) PairsRatio: 21.63 closes: https://github.com/official-stockfish/Stockfish/pull/4163 Bench: 4425574
Copyright 2011–2024 Next Chess Move LLC