Dev Builds » 20230122-0954

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:01:11 1224269 1654 690 170 794 +113.06 ± 7.24 0 20 276 522 9 +250.71 ± 20.48
ncm-dbt-02 04:58:02 1230758 1642 680 161 801 +113.71 ± 7.81 0 30 260 513 18 +246.43 ± 21.17
ncm-dbt-03 05:01:34 1236783 1676 673 178 825 +105.76 ± 7.72 1 30 296 495 16 +226.41 ± 19.81
ncm-dbt-04 05:01:28 1229007 1674 665 163 846 +107.49 ± 7.7 0 29 296 493 19 +228.63 ± 19.8
ncm-dbt-05 04:59:46 1212253 1664 699 163 802 +116.04 ± 7.71 0 22 277 508 25 +248.63 ± 20.46
ncm-dbt-06 05:01:21 1225854 1690 687 160 843 +112.07 ± 7.76 0 31 277 516 21 +240.18 ± 20.5
ncm-et-3 06:27:41 1297717 1662 720 172 770 +119.0 ± 7.73 0 23 262 521 25 +257.23 ± 21.07
ncm-et-4 06:27:15 1300488 1666 662 193 811 +100.52 ± 8.15 0 45 295 472 21 +208.84 ± 19.86
ncm-et-9 06:27:42 1291092 1668 697 161 810 +115.75 ± 7.96 0 33 256 521 24 +248.48 ± 21.33
ncm-et-10 06:28:19 1285581 1670 695 160 815 +115.37 ± 7.66 1 22 274 517 21 +250.07 ± 20.58
ncm-et-13 06:27:04 1304854 1678 692 168 818 +112.24 ± 7.83 1 29 275 513 21 +241.21 ± 20.57
ncm-et-15 06:27:48 1304139 1656 703 156 797 +119.23 ± 7.88 1 23 259 518 27 +257.15 ± 21.2
20000 8263 2005 9732 +112.48 ± 2.24 4 337 3303 6109 247 +241.64 ± 5.92

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
181185 ncm-dbt-02 1242190 142 58 17 67 +103.25 ± 26.39 0 3 25 42 1 +221.58 ± 69.86
181184 ncm-dbt-01 1240448 154 64 18 72 +107.04 ± 24.09 0 2 28 46 1 +232.48 ± 65.54
181183 ncm-dbt-05 1200708 164 69 18 77 +111.74 ± 24.06 0 2 29 49 2 +239.5 ± 64.41
181182 ncm-dbt-04 1208867 174 70 20 84 +102.72 ± 24.66 0 4 31 50 2 +215.71 ± 62.4
181181 ncm-dbt-03 1236742 176 66 24 86 +84.54 ± 22.49 0 4 38 46 0 +180.47 ± 55.64
181180 ncm-dbt-06 1234154 190 81 17 92 +121.78 ± 23.23 0 3 28 61 3 +264.66 ± 66.09
181179 ncm-dbt-02 1211005 500 203 44 253 +114.45 ± 13.89 0 10 74 163 3 +254.16 ± 39.94
181178 ncm-dbt-05 1199063 500 217 47 236 +123.02 ± 14.49 0 7 76 157 10 +263.42 ± 39.41
181177 ncm-dbt-01 1232077 500 205 48 247 +112.91 ± 13.56 0 6 86 153 5 +245.2 ± 36.85
181176 ncm-dbt-03 1232456 500 206 56 238 +107.54 ± 14.83 0 10 89 142 9 +221.9 ± 36.31
181175 ncm-dbt-04 1228605 500 187 44 269 +102.22 ± 14.18 0 11 89 146 4 +217.85 ± 36.33
181174 ncm-dbt-06 1229707 500 204 38 258 +119.89 ± 14.83 0 9 76 155 10 +254.16 ± 39.42
181173 ncm-dbt-02 1231543 500 211 58 231 +109.83 ± 14.37 0 8 89 145 8 +230.16 ± 36.26
181172 ncm-dbt-05 1239755 500 211 51 238 +115.22 ± 14.37 0 5 91 143 11 +238.66 ± 35.65
181171 ncm-dbt-01 1217670 500 201 46 253 +111.37 ± 12.87 0 7 81 162 0 +251.89 ± 38.11
181170 ncm-dbt-03 1241637 500 200 46 254 +110.6 ± 14.84 1 11 76 157 5 +240.82 ± 39.35
181169 ncm-dbt-06 1202954 500 198 45 257 +109.83 ± 13.89 0 9 83 154 4 +238.66 ± 37.66
181168 ncm-dbt-04 1222366 500 193 52 255 +100.7 ± 14.32 0 10 95 139 6 +209.91 ± 35.06
181167 ncm-dbt-05 1209489 500 202 47 251 +111.37 ± 13.39 0 8 81 159 2 +247.41 ± 38.13
181166 ncm-dbt-02 1238295 500 208 42 250 +119.89 ± 14.19 0 9 72 163 6 +263.42 ± 40.52
181165 ncm-dbt-01 1206882 500 220 58 222 +116.77 ± 13.01 0 5 81 161 3 +261.07 ± 38.02
181164 ncm-dbt-03 1236300 500 201 52 247 +106.77 ± 12.88 0 5 93 150 2 +234.38 ± 35.22
181163 ncm-dbt-06 1236603 500 204 60 236 +102.97 ± 14.02 0 10 90 146 4 +219.87 ± 36.1
181162 ncm-dbt-04 1256191 500 215 47 238 +121.45 ± 13.49 0 4 81 158 7 +265.78 ± 37.95
166406 ncm-et-15 1299557 156 67 16 73 +117.91 ± 22.79 0 2 23 53 0 +271.69 ± 73.38
166405 ncm-et-4 1307721 166 68 23 75 +96.59 ± 25.34 0 4 32 45 2 +199.32 ± 61.23
166404 ncm-et-9 1307590 168 66 16 86 +106.62 ± 23.68 0 3 29 51 1 +231.91 ± 64.64
166403 ncm-et-3 1290288 162 68 18 76 +110.84 ± 25.14 0 3 27 49 2 +236.83 ± 67.19
166402 ncm-et-10 1269535 170 66 16 88 +105.29 ± 24.3 0 4 28 52 1 +228.32 ± 65.89
166401 ncm-et-13 1312265 178 75 15 88 +121.87 ± 27.22 0 5 24 55 5 +250.75 ± 71.0
166400 ncm-et-3 1292864 500 222 55 223 +120.67 ± 15.3 0 9 78 150 13 +249.64 ± 38.9
166399 ncm-et-4 1297411 500 202 55 243 +105.25 ± 14.04 0 11 84 152 3 +228.08 ± 37.44
166398 ncm-et-10 1293292 500 203 58 239 +103.73 ± 14.65 0 10 93 139 8 +213.85 ± 35.47
166397 ncm-et-13 1306285 500 207 53 240 +110.6 ± 13.73 0 9 81 157 3 +243.0 ± 38.14
166396 ncm-et-15 1308440 500 208 53 239 +111.37 ± 13.56 0 3 97 142 8 +234.38 ± 34.19
166395 ncm-et-9 1278547 500 214 54 232 +115.22 ± 14.21 0 12 69 166 3 +256.44 ± 41.25
166394 ncm-et-3 1309023 500 220 45 235 +126.97 ± 13.22 0 4 73 167 6 +285.49 ± 40.16
166393 ncm-et-4 1297614 500 199 53 248 +104.49 ± 14.66 0 11 89 143 7 +217.85 ± 36.33
166392 ncm-et-10 1287847 500 215 45 240 +123.02 ± 12.74 0 2 81 162 5 +275.45 ± 37.75
166391 ncm-et-13 1285921 500 205 47 248 +113.68 ± 13.72 1 2 92 148 7 +245.2 ± 35.25
166390 ncm-et-15 1301182 500 220 44 236 +127.76 ± 15.25 1 7 69 161 12 +275.45 ± 41.43
166389 ncm-et-9 1290027 500 209 41 250 +121.45 ± 15.45 0 10 75 152 13 +251.89 ± 39.67
166388 ncm-et-13 1314947 500 205 53 242 +109.07 ± 14.84 0 13 78 153 6 +232.26 ± 38.82
166387 ncm-et-9 1288204 500 208 50 242 +113.68 ± 14.22 0 8 83 152 7 +243.0 ± 37.64
166386 ncm-et-4 1299207 500 193 62 245 +93.2 ± 15.92 0 19 90 132 9 +185.33 ± 36.06
166385 ncm-et-3 1298693 500 210 54 236 +112.14 ± 13.56 0 7 84 155 4 +245.2 ± 37.37
166384 ncm-et-15 1307377 500 208 43 249 +119.11 ± 14.68 0 11 70 162 7 +258.75 ± 41.02
166383 ncm-et-10 1291653 500 211 41 248 +123.02 ± 14.32 1 6 72 164 7 +273.0 ± 40.54

Commit

Commit ID a08b8d4e9711c20acedbfe17d618c3c384b339ec
Author Joost VandeVondele
Date 2023-01-22 09:54:15 UTC
Update UCI_Elo parameterization The old parameterization (https://github.com/official-stockfish/Stockfish/pull/2225/files) has now become quite inaccurate. This updates the formula based on updated results with master. The formula is based on a fit of the Elo results for games played between master at various skill levels, and various versions of the Stash engine, which have been ranked at CCRL. ``` # PLAYER : RATING ERROR POINTS PLAYED (%) 1 master-skill-19 : 3191.1 40.4 940.0 1707 55 2 master-skill-18 : 3170.3 39.3 1343.0 2519 53 3 master-skill-17 : 3141.3 37.8 2282.0 4422 52 4 master-skill-16 : 3111.2 37.1 2773.0 5423 51 5 master-skill-15 : 3069.5 37.2 2728.5 5386 51 6 master-skill-14 : 3024.8 36.1 2702.0 5339 51 7 master-skill-13 : 2972.9 35.4 2645.5 5263 50 8 master-skill-12 : 2923.1 35.0 2653.5 5165 51 9 master-skill-11 : 2855.5 33.6 2524.0 5081 50 10 master-skill-10 : 2788.3 32.0 2724.5 5511 49 11 stash-bot-v25.0 : 2744.0 31.5 1952.5 3840 51 12 master-skill-9 : 2702.8 30.5 2670.0 5018 53 13 master-skill-8 : 2596.2 28.5 2669.5 4975 54 14 stash-bot-v21.0 : 2561.2 30.0 1338.0 3366 40 15 master-skill-7 : 2499.5 28.5 1934.0 4178 46 16 stash-bot-v20.0 : 2452.6 27.7 1606.5 3378 48 17 stash-bot-v19.0 : 2425.3 26.7 1787.0 3365 53 18 master-skill-6 : 2363.2 26.4 2510.5 4379 57 19 stash-bot-v17.0 : 2280.7 25.4 2209.0 4378 50 20 master-skill-5 : 2203.7 25.3 2859.5 5422 53 21 stash-bot-v15.3 : 2200.0 25.4 1757.0 4383 40 22 stash-bot-v14 : 2145.9 25.5 2890.0 5167 56 23 stash-bot-v13 : 2042.7 25.8 2263.5 4363 52 24 stash-bot-v12 : 1963.4 25.8 1769.5 4210 42 25 master-skill-4 : 1922.9 25.9 2690.0 5399 50 26 stash-bot-v11 : 1873.0 26.3 2203.5 4335 51 27 stash-bot-v10 : 1783.8 27.8 2568.5 4301 60 28 master-skill-3 : 1742.3 27.8 1909.5 4439 43 29 master-skill-2 : 1608.4 29.4 2064.5 4389 47 30 stash-bot-v9 : 1582.6 30.2 2130.0 4230 50 31 master-skill-1 : 1467.6 31.3 2015.5 4244 47 32 stash-bot-v8 : 1452.8 31.5 1953.5 3780 52 33 master-skill-0 : 1320.1 32.9 651.5 2083 31 ``` Skill 0 .. 19, now covers CCRL Blitz Elo from 1320 to 3190, approximately. Indeed, the Elo of stash in this analysis is only to within +- 100 Elo of CCRL, probably because it depends quite a bit on the opponent pool. To obtain a skill level for a given Elo number, the above data is fit as a 3rd degree polynomial Skill(Elo). A quick test confirms the correspondence to the above table: ``` Score of master-elo-2721 vs stash-bot-v21.0: 51 - 16 - 19 [0.703] 86 Elo difference: 150.1 +/- 70.2, LOS: 100.0 %, DrawRatio: 22.1 % ``` closes https://github.com/official-stockfish/Stockfish/pull/4341 No functional change.
Copyright 2011–2024 Next Chess Move LLC