Dev Builds » 20230701-1059

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:55:59 1200904 3326 1435 304 1587 +123.04 ± 5.42 1 46 481 1091 44 +271.93 ± 15.55
ncm-dbt-02 09:52:22 1234104 3320 1431 288 1601 +124.71 ± 5.41 1 42 477 1093 47 +275.97 ± 15.61
ncm-dbt-03 09:56:39 1223543 3324 1424 289 1611 +123.6 ± 5.39 1 41 488 1086 46 +272.9 ± 15.43
ncm-dbt-04 09:56:46 1224470 3352 1417 289 1646 +121.66 ± 5.09 0 36 502 1112 26 +273.93 ± 15.2
ncm-dbt-05 09:52:06 1237846 3328 1445 292 1591 +125.57 ± 5.37 0 42 475 1099 48 +277.96 ± 15.64
ncm-dbt-06 09:56:11 1229014 3350 1422 296 1632 +121.5 ± 5.32 1 48 484 1108 34 +270.9 ± 15.5
20000 8574 1758 9668 +123.34 ± 2.18 4 255 2907 6589 245 +273.92 ± 6.31

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
196014 ncm-dbt-02 1231399 320 137 24 159 +128.2 ± 17.07 0 3 46 106 5 +284.85 ± 50.97
196013 ncm-dbt-05 1243470 328 137 32 159 +115.27 ± 17.6 0 4 57 97 6 +242.82 ± 45.44
196012 ncm-dbt-03 1245112 324 140 28 156 +125.26 ± 17.65 1 2 48 106 5 +279.59 ± 49.82
196011 ncm-dbt-01 1204556 326 134 37 155 +106.6 ± 17.36 0 8 51 103 1 +234.89 ± 48.26
196010 ncm-dbt-04 1217030 352 157 32 163 +128.99 ± 16.01 0 3 50 118 5 +289.24 ± 48.82
196009 ncm-dbt-06 1225680 350 142 26 182 +119.67 ± 17.11 0 7 49 115 4 +263.42 ± 49.3
196008 ncm-dbt-02 1239752 500 198 43 259 +111.37 ± 13.89 0 8 84 153 5 +240.82 ± 37.4
196007 ncm-dbt-03 1226902 500 213 49 238 +118.33 ± 13.35 0 6 78 162 4 +263.42 ± 38.85
196006 ncm-dbt-05 1268575 500 228 38 234 +138.99 ± 13.48 0 2 67 170 11 +312.48 ± 41.9
196005 ncm-dbt-01 1204969 500 205 47 248 +113.68 ± 13.38 1 3 87 155 4 +251.89 ± 36.48
196004 ncm-dbt-06 1242154 500 215 57 228 +113.68 ± 14.22 1 8 77 160 4 +251.89 ± 39.16
196003 ncm-dbt-04 1208840 500 208 46 246 +116.77 ± 12.83 0 5 80 163 2 +263.42 ± 38.27
196002 ncm-dbt-02 1249770 500 215 46 239 +122.24 ± 13.48 1 4 74 167 4 +277.93 ± 39.92
196001 ncm-dbt-05 1232697 500 226 42 232 +134.15 ± 13.97 0 5 66 169 10 +298.62 ± 42.41
196000 ncm-dbt-03 1211170 500 218 41 241 +128.55 ± 14.08 0 5 73 162 10 +280.42 ± 40.21
195999 ncm-dbt-01 1198352 500 219 38 243 +131.74 ± 14.2 0 8 61 173 8 +295.94 ± 44.08
195998 ncm-dbt-06 1228066 500 215 51 234 +118.33 ± 13.86 0 5 84 153 8 +254.16 ± 37.27
195997 ncm-dbt-04 1215220 500 207 43 250 +118.33 ± 13.35 0 6 78 162 4 +263.42 ± 38.85
195996 ncm-dbt-05 1210086 500 214 36 250 +129.35 ± 13.9 0 8 62 174 6 +293.29 ± 43.72
195995 ncm-dbt-02 1242133 500 218 33 249 +134.95 ± 13.41 0 4 65 173 8 +306.84 ± 42.73
195994 ncm-dbt-03 1224859 500 216 36 248 +130.94 ± 13.51 0 5 67 171 7 +295.94 ± 42.08
195993 ncm-dbt-06 1219711 500 206 43 251 +117.55 ± 13.0 0 7 74 168 1 +268.17 ± 39.97
195992 ncm-dbt-04 1219638 500 215 38 247 +128.55 ± 13.19 0 4 71 169 6 +290.66 ± 40.76
195991 ncm-dbt-01 1208701 500 210 41 249 +122.24 ± 13.66 0 7 72 166 5 +273.0 ± 40.54
195990 ncm-dbt-05 1237059 500 207 53 240 +110.6 ± 14.84 0 11 82 149 8 +232.26 ± 37.9
195989 ncm-dbt-02 1222997 500 224 48 228 +127.76 ± 14.1 0 8 65 170 7 +285.49 ± 42.7
195988 ncm-dbt-06 1231400 500 218 41 241 +128.55 ± 13.19 0 6 65 175 4 +295.94 ± 42.75
195987 ncm-dbt-03 1209611 500 221 43 236 +129.35 ± 13.54 0 5 69 169 7 +290.66 ± 41.43
195986 ncm-dbt-04 1240437 500 206 50 244 +112.14 ± 13.04 0 5 87 155 3 +247.41 ± 36.56
195985 ncm-dbt-01 1157340 500 227 40 233 +136.56 ± 14.1 0 7 58 176 9 +309.64 ± 45.27
195984 ncm-dbt-05 1231754 500 220 39 241 +131.74 ± 13.12 0 6 61 179 4 +306.84 ± 44.17
195983 ncm-dbt-02 1226061 500 216 46 238 +123.02 ± 14.33 0 7 75 159 9 +265.78 ± 39.69
195982 ncm-dbt-04 1225389 500 205 46 249 +114.45 ± 13.38 0 8 77 163 2 +256.44 ± 39.15
195981 ncm-dbt-03 1218034 500 205 47 248 +113.68 ± 13.89 0 8 81 156 5 +247.41 ± 38.13
195980 ncm-dbt-06 1235064 500 209 36 255 +125.38 ± 13.61 0 8 65 173 4 +285.49 ± 42.7
195979 ncm-dbt-01 1226220 500 219 44 237 +126.97 ± 12.85 0 2 77 165 6 +285.49 ± 38.83
195978 ncm-dbt-05 1241282 500 213 52 235 +116.0 ± 13.2 0 6 80 161 3 +258.75 ± 38.32
195977 ncm-dbt-02 1226620 500 223 48 229 +126.97 ± 14.45 0 8 68 165 9 +277.93 ± 41.74
195976 ncm-dbt-06 1221025 500 217 42 241 +126.97 ± 14.28 0 7 70 164 9 +277.93 ± 41.14
195975 ncm-dbt-01 1206193 500 221 57 222 +118.33 ± 15.31 0 11 75 153 11 +247.41 ± 39.65
195974 ncm-dbt-03 1229113 500 211 45 244 +119.89 ± 14.68 0 10 72 160 8 +258.75 ± 40.49
195973 ncm-dbt-04 1244742 500 219 34 247 +134.95 ± 12.84 0 5 59 182 4 +318.25 ± 44.96

Commit

Commit ID ef94f77f8c827a2395f1c40f53311a3b1f20bc5b
Author Daniel Monroe
Date 2023-07-01 10:59:28 UTC
Update default net to nn-a3d1bfca1672.nnue faster permutation of master net weights Activation data taken from https://drive.google.com/drive/folders/1Ec9YuuRx4N03GPnVPoQOW70eucOKngQe?usp=sharing Permutation found using https://github.com/Ergodice/nnue-pytorch/blob/836387a0e5e690431d404158c46648710f13904d/ftperm.py See also https://github.com/glinscott/nnue-pytorch/pull/254 The algorithm greedily selects 2- and 3-cycles that can be permuted to increase the number of runs of zeroes. The percent of zero runs from the master net increased from 68.46 to 70.11 from 2-cycles and only increased to 70.32 when considering 3-cycles. Interestingly, allowing both halves of L1 to intermix when creating zero runs can give another 0.5% zero-run density increase with this method. Measured speedup: ``` CPU: 16 x AMD Ryzen 9 3950X 16-Core Processor Result of 50 runs base (./stockfish.master ) = 1561556 +/- 5439 test (./stockfish.patch ) = 1575788 +/- 5427 diff = +14231 +/- 2636 speedup = +0.0091 P(speedup > 0) = 1.0000 ``` closes https://github.com/official-stockfish/Stockfish/pull/4640 No functional change
Copyright 2011–2024 Next Chess Move LLC