Dev Builds » 20211126-1716

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 10:01:58 1296860 3346 1114 568 1664 +57.21 ± 5.85 3 184 764 708 14 +115.14 ± 12.32
ncm-dbt-02 09:58:49 1302753 3306 1102 554 1650 +58.13 ± 5.89 4 176 756 702 15 +117.11 ± 12.39
ncm-dbt-03 10:04:07 1323561 3328 1061 529 1738 +56.02 ± 5.7 1 167 813 665 18 +111.17 ± 11.91
ncm-dbt-04 10:04:16 1316859 3364 1096 586 1682 +53.08 ± 5.87 7 189 786 687 13 +107.4 ± 12.15
ncm-dbt-05 09:57:37 967971 3326 1063 558 1705 +53.16 ± 5.78 1 189 789 672 12 +106.41 ± 12.12
ncm-dbt-06 10:02:55 1315747 3330 1096 558 1676 +56.63 ± 5.76 5 169 784 697 10 +115.27 ± 12.15
20000 6532 3353 10115 +55.7 ± 2.37 21 1074 4692 4131 82 +112.06 ± 4.97

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
199456 ncm-dbt-02 1198520 138 47 14 77 +84.72 ± 28.37 0 4 30 33 2 +168.09 ± 63.01 ↓
199455 ncm-dbt-03 1247234 166 48 31 87 +35.7 ± 27.41 0 14 39 29 1 +67.82 ± 55.01 ↓
199454 ncm-dbt-05 1224646 164 52 25 87 +57.72 ± 24.23 0 6 44 31 1 +114.09 ± 50.83 ↓
199453 ncm-dbt-01 1208633 172 50 31 91 +38.54 ± 22.99 0 9 49 28 0 +78.05 ± 48.03 ↓
199452 ncm-dbt-06 1231662 172 57 33 82 +48.8 ± 27.83 1 12 35 38 0 +104.0 ± 57.86 ↓
199451 ncm-dbt-04 1249606 188 61 35 92 +48.36 ± 25.31 0 13 43 37 1 +94.68 ± 52.39 ↓
199450 ncm-dbt-02 1192946 500 160 89 251 +49.67 ± 15.29 0 33 115 100 2 +98.44 ± 31.86 ↓
199449 ncm-dbt-05 1236888 500 170 87 243 +58.21 ± 15.57 1 30 105 113 1 +119.89 ± 33.31 ↓
199448 ncm-dbt-03 1238352 500 159 84 257 +52.51 ± 16.41 0 35 113 94 8 +95.44 ± 32.13 ↓
199447 ncm-dbt-06 1225779 500 158 84 258 +51.8 ± 14.35 0 26 125 98 1 +104.49 ± 30.42 ↓
199446 ncm-dbt-01 1198275 500 159 98 243 +42.6 ± 15.66 1 36 116 95 2 +85.04 ± 31.71 ↓
199445 ncm-dbt-04 1214469 500 145 102 253 +29.95 ± 15.26 3 35 128 84 0 +64.66 ± 30.12 ↓
199444 ncm-dbt-02 1197059 500 166 98 236 +47.54 ± 14.92 0 30 125 92 3 +92.46 ± 30.47 ↓
199443 ncm-dbt-06 1224418 500 159 81 260 +54.64 ± 14.49 0 25 124 99 2 +109.07 ± 30.54 ↓
199442 ncm-dbt-01 1185785 500 167 83 250 +58.93 ± 15.35 2 26 108 114 0 +124.6 ± 32.89 ↓
199441 ncm-dbt-05 1238534 500 154 88 258 +46.13 ± 14.18 0 27 131 91 1 +92.46 ± 29.6 ↓
199440 ncm-dbt-03 1238500 500 171 74 255 +68.27 ± 14.54 0 22 110 117 1 +140.62 ± 32.58 ↓
199439 ncm-dbt-04 1224477 500 180 83 237 +68.27 ± 14.54 0 20 116 111 3 +137.37 ± 31.63 ↓
199438 ncm-dbt-02 1189628 500 173 72 255 +71.16 ± 14.92 0 22 108 117 3 +143.89 ± 32.9 ↓
199437 ncm-dbt-01 1206913 500 168 94 238 +51.8 ± 14.62 0 28 121 100 1 +104.49 ± 31.0 ↓
199436 ncm-dbt-06 1220321 500 164 86 250 +54.65 ± 14.76 1 25 120 103 1 +112.14 ± 31.13 ↓
199435 ncm-dbt-04 1209427 500 167 82 251 +59.64 ± 14.46 0 23 121 104 2 +119.89 ± 30.94 ↓
199434 ncm-dbt-03 1231174 500 164 76 260 +61.79 ± 13.99 0 19 126 103 2 +124.6 ± 30.12 ↓
199433 ncm-dbt-05 1225453 500 156 79 265 +53.93 ± 14.32 0 25 124 100 1 +109.07 ± 30.54 ↓
199432 ncm-dbt-02 1195737 500 168 77 255 +63.95 ± 15.04 1 24 109 115 1 +132.54 ± 32.75 ↓
199431 ncm-dbt-01 1185714 500 177 75 248 +71.88 ± 15.34 0 23 107 115 5 +142.26 ± 33.06 ↓
199430 ncm-dbt-06 1225846 500 171 82 247 +62.51 ± 14.72 1 20 121 105 3 +126.18 ± 30.9 ↓
199429 ncm-dbt-04 1225788 500 159 100 241 +41.19 ± 15.7 2 34 119 93 2 +83.57 ± 31.31 ↓
199428 ncm-dbt-05 1233945 500 166 75 259 +63.94 ± 14.91 0 23 117 106 4 +126.18 ± 31.53 ↓
199427 ncm-dbt-03 1232034 500 148 72 280 +53.22 ± 13.44 0 20 134 96 0 +109.07 ± 29.01 ↓
199426 ncm-dbt-02 1198623 500 167 87 246 +56.07 ± 14.56 1 24 119 106 0 +116.78 ± 31.26 ↓
199425 ncm-dbt-01 1210849 500 160 78 262 +57.5 ± 14.63 0 25 120 103 2 +115.23 ± 31.11 ↓
199424 ncm-dbt-06 1216593 500 169 88 243 +56.79 ± 15.38 1 30 106 113 0 +118.33 ± 33.16 ↓
199423 ncm-dbt-04 1228077 500 163 77 260 +60.36 ± 14.62 1 22 118 108 1 +124.6 ± 31.38 ↓
199422 ncm-dbt-03 1219072 500 156 78 266 +54.64 ± 14.89 0 25 127 93 5 +104.49 ± 30.12 ↓
199421 ncm-dbt-05 1231124 500 156 83 261 +51.09 ± 14.98 0 31 116 102 1 +102.97 ± 31.72 ↓
199420 ncm-dbt-01 1166003 500 175 77 248 +68.99 ± 15.63 0 28 99 120 3 +138.99 ± 34.28 ↓
199419 ncm-dbt-02 1220822 500 172 89 239 +58.21 ± 15.82 2 26 113 105 4 +116.78 ± 32.15 ↓
199418 ncm-dbt-05 1242664 500 157 103 240 +37.67 ± 15.5 0 40 118 90 2 +73.34 ± 31.44 ↓
199417 ncm-dbt-06 1214125 500 172 80 248 +64.66 ± 14.94 1 21 116 109 3 +130.94 ± 31.66 ↓
199416 ncm-dbt-03 1218308 500 162 85 253 +53.93 ± 14.46 1 23 125 100 1 +110.6 ± 30.39 ↓
199415 ncm-dbt-04 1231777 500 164 79 257 +59.64 ± 15.64 0 32 103 113 2 +119.89 ± 33.6 ↓
176679 ncm-dbt-06 1480449 8 3 0 5 +136.8 ± 89.09 0 0 1 3 0 +337.69 ± 541.44 ↓
176678 ncm-dbt-05 503688 12 6 1 5 +154.01 ± 169.76 0 1 0 4 1 +279.52 ± 587.7 ↓
176677 ncm-dbt-03 1480436 12 4 2 6 +58.39 ± 68.56 0 0 4 2 0 +120.33 ± 161.99 ↓
176676 ncm-dbt-02 1482579 18 4 5 9 -19.32 ± 85.67 0 3 4 2 0 -38.76 ± 185.98 ↓
176675 ncm-dbt-01 1477985 24 6 6 12 0.0 ± 57.31 0 2 8 2 0 -0.0 ± 117.81 ↓
176674 ncm-dbt-04 1469345 26 11 5 10 +81.63 ± 64.15 0 1 5 7 0 +173.46 ± 173.59 ↓
176673 ncm-dbt-03 1484577 50 16 9 25 +48.96 ± 46.46 0 3 12 10 0 +99.95 ± 101.68 ↓
176672 ncm-dbt-05 504007 50 18 6 26 +85.04 ± 41.74 0 1 11 13 0 +181.7 ± 107.11 ↓
176671 ncm-dbt-06 1473147 50 13 4 33 +63.22 ± 48.63 0 3 10 12 0 +130.94 ± 112.59 ↓
176670 ncm-dbt-01 1470857 50 18 6 26 +85.02 ± 51.0 0 2 10 12 1 +164.07 ± 113.84 ↓
176669 ncm-dbt-02 1483500 50 14 6 30 +56.07 ± 38.31 0 1 15 9 0 +115.23 ± 84.94 ↓
176668 ncm-dbt-04 1475855 50 17 7 26 +70.42 ± 60.89 0 4 9 10 2 +115.23 ± 116.78 ↓
176667 ncm-dbt-03 1486446 50 16 8 26 +56.07 ± 47.61 0 3 11 11 0 +115.23 ± 106.91 ↓
176666 ncm-dbt-05 503369 50 13 5 32 +56.07 ± 51.66 0 4 9 12 0 +115.23 ± 116.78 ↓
176665 ncm-dbt-06 1478598 50 14 9 27 +34.86 ± 43.75 0 3 14 8 0 +70.44 ± 92.08 ↓
176664 ncm-dbt-02 1489057 50 15 5 30 +70.43 ± 45.16 0 2 11 12 0 +147.19 ± 107.43 ↓
176663 ncm-dbt-01 1479058 50 17 7 26 +70.43 ± 45.16 0 2 11 12 0 +147.19 ± 107.43 ↓
176662 ncm-dbt-04 1482141 50 15 9 26 +41.91 ± 56.76 1 3 10 11 0 +99.95 ± 111.26 ↓
176661 ncm-dbt-05 503369 50 15 6 29 +63.22 ± 39.41 0 1 14 10 0 +130.94 ± 89.9 ↓
176660 ncm-dbt-03 1483039 50 17 10 23 +48.96 ± 46.46 0 3 12 10 0 +99.95 ± 101.68 ↓
176659 ncm-dbt-06 1482285 50 16 11 23 +34.86 ± 47.98 0 4 12 9 0 +70.44 ± 101.42 ↓
176658 ncm-dbt-01 1475395 50 17 13 20 +27.85 ± 42.16 0 3 15 7 0 +56.07 ± 87.47 ↓
176657 ncm-dbt-02 1481812 50 16 12 22 +27.85 ± 57.65 0 7 7 11 0 +56.07 ± 121.58 ↓
176655 ncm-dbt-04 1474487 50 14 7 29 +48.96 ± 42.0 0 2 14 9 0 +99.95 ± 91.41 ↓

Commit

Commit ID 9ee58dc7a75d7cdc09935f8782e927bb19700afe
Author Joost VandeVondele
Date 2021-11-26 17:16:04 UTC
Update default net to nn-3678835b1d3d.nnue New net trained with nnue-pytorch, started from the master net on a data set of Leela (T60.binpack+T74.binpck) and Stockfish data (wrongIsRight_nodes5000pv2.binpack), available as a single interleaved binpack: https://drive.google.com/file/d/12uWZIA3F2cNbraAzQNb1jgf3tq_6HkTr/view?usp=sharing The nnue-pytorch branch used is https://github.com/vondele/nnue-pytorch/tree/wdl, which has the new feature to filter positions based on the likelihood of the current evaluation leading to the game outcome. It should make it less likely to try to learn from misevaluated positions. Standard options have been used, starting from the master net: --gpus 1 --threads 4 --num-workers 4 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 12 --features=HalfKAv2_hm^ --lambda=1.0 Testing with games shows neutral Elo at STC, and good performance at LTC: STC: https://tests.stockfishchess.org/tests/view/619eb597c0a4ea18ba95a4dc ELO: -0.44 +-1.8 (95%) LOS: 31.2% Total: 40000 W: 10447 L: 10498 D: 19055 Ptnml(0-2): 254, 4576, 10260, 4787, 123 LTC: https://tests.stockfishchess.org/tests/view/619f6e87c0a4ea18ba95a53f ELO: 3.30 +-1.8 (95%) LOS: 100.0% Total: 33062 W: 8560 L: 8246 D: 16256 Ptnml(0-2): 54, 3358, 9352, 3754, 13 passed LTC SPRT: https://tests.stockfishchess.org/tests/view/61a0864e8967bbf894416e65 LLR: 2.94 (-2.94,2.94) <0.50,3.00> Total: 29376 W: 7663 L: 7396 D: 14317 Ptnml(0-2): 67, 3017, 8205, 3380, 19 closes https://github.com/official-stockfish/Stockfish/pull/3808 Bench: 7011501
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version