Dev Builds » 20220514-1047

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 10:23:30 1220261 3414 1342 419 1653 +96.33 ± 5.79 2 100 622 939 44 +198.39 ± 13.66
ncm-dbt-02 10:24:38 1312733 3426 1331 389 1706 +98.05 ± 5.65 1 88 634 948 42 +203.1 ± 13.53
ncm-dbt-03 09:34:53 1194613 3194 1238 390 1566 +94.51 ± 5.97 2 101 574 887 33 +196.26 ± 14.22
ncm-dbt-04 10:26:19 1268462 3414 1357 400 1657 +100.07 ± 5.4 0 65 656 950 36 +209.68 ± 13.26
ncm-dbt-05 10:22:00 1281621 3390 1336 389 1665 +99.71 ± 5.48 0 80 615 973 27 +211.27 ± 13.74
ncm-dbt-06 09:33:19 1120614 3162 1215 407 1540 +90.79 ± 5.87 0 98 606 848 29 +187.5 ± 13.84
20000 7819 2394 9787 +96.66 ± 2.32 5 532 3707 5545 211 +201.15 ± 5.59

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
203888 ncm-dbt-06 626463 124 47 22 55 +71.01 ± 27.47 0 3 32 26 1 +141.89 ± 59.63 ↓
203887 ncm-dbt-04 812361 128 48 16 64 +88.73 ± 33.21 0 5 26 29 4 +162.99 ± 68.13 ↓
203886 ncm-dbt-01 623367 124 45 17 62 +79.82 ± 35.51 0 7 24 27 4 +141.89 ± 70.61 ↓
203885 ncm-dbt-03 875614 160 59 23 78 +79.53 ± 25.33 0 6 32 42 0 +168.4 ± 61.19 ↓
203884 ncm-dbt-02 1196566 170 71 23 76 +100.84 ± 27.91 1 2 35 42 5 +199.12 ± 58.06 ↓
203883 ncm-dbt-05 832311 168 74 20 74 +115.77 ± 25.35 0 3 27 51 3 +244.73 ± 67.25 ↓
203882 ncm-dbt-06 809233 24 9 5 10 +58.42 ± 76.7 0 1 7 3 1 +88.74 ± 131.95 ↓
203881 ncm-dbt-01 781473 30 12 4 14 +94.91 ± 69.06 0 1 6 7 1 +175.73 ± 154.46 ↓
203880 ncm-dbt-04 1233934 42 16 6 20 +84.33 ± 46.59 0 1 9 11 0 +179.99 ± 120.55 ↓
203879 ncm-dbt-06 1222716 14 4 3 7 +24.85 ± 84.38 0 1 4 2 0 +49.98 ± 184.5 ↓
203878 ncm-dbt-01 1219164 10 3 1 6 +70.36 ± 79.51 0 0 3 2 0 +147.07 ± 211.27 ↓
203877 ncm-dbt-03 1237127 34 13 6 15 +72.56 ± 52.2 0 1 8 8 0 +152.08 ± 127.51 ↓
203876 ncm-dbt-05 1244919 472 191 49 232 +107.86 ± 14.77 0 10 79 142 5 +230.43 ± 38.62 ↓
203875 ncm-dbt-02 1196033 500 200 52 248 +106.01 ± 14.2 0 8 93 142 7 +221.9 ± 35.4 ↓
203874 ncm-dbt-04 1230119 500 206 63 231 +102.22 ± 13.54 0 9 91 148 2 +221.9 ± 35.86 ↓
203873 ncm-dbt-06 1243025 500 187 70 243 +82.83 ± 15.41 0 20 99 125 6 +165.8 ± 34.39 ↓
203872 ncm-dbt-03 1251273 500 201 65 234 +96.94 ± 15.72 1 17 83 143 6 +202.15 ± 37.5 ↓
203871 ncm-dbt-01 1207756 500 202 64 234 +98.44 ± 15.6 1 12 95 132 10 +198.34 ± 35.12 ↓
203870 ncm-dbt-05 1213874 500 186 49 265 +97.69 ± 14.43 0 12 94 139 5 +204.07 ± 35.31 ↓
203869 ncm-dbt-02 1196570 500 192 60 248 +93.95 ± 14.07 0 9 106 129 6 +192.71 ± 32.93 ↓
203868 ncm-dbt-04 1228940 500 206 51 243 +111.37 ± 13.73 0 6 89 149 6 +238.66 ± 36.16 ↓
203867 ncm-dbt-06 1250229 500 192 60 248 +93.95 ± 15.39 0 18 88 138 6 +192.71 ± 36.47 ↓
203866 ncm-dbt-03 1254477 500 194 67 239 +90.22 ± 14.75 0 11 110 120 9 +178.11 ± 32.33 ↓
203865 ncm-dbt-01 1227274 500 204 55 241 +106.77 ± 14.67 0 11 86 146 7 +223.94 ± 36.98 ↓
203864 ncm-dbt-05 1227399 500 188 53 259 +96.19 ± 14.56 0 13 94 138 5 +200.24 ± 35.32 ↓
203863 ncm-dbt-02 1198135 500 192 56 252 +96.94 ± 15.16 0 16 88 140 6 +200.24 ± 36.52 ↓
203862 ncm-dbt-04 1251605 500 191 63 246 +90.96 ± 13.87 0 10 106 130 4 +189.0 ± 32.98 ↓
203861 ncm-dbt-06 1211616 500 192 70 238 +86.52 ± 14.67 0 16 100 130 4 +178.11 ± 34.22 ↓
203860 ncm-dbt-03 1220456 500 197 58 245 +99.2 ± 15.75 1 16 83 143 7 +206.01 ± 37.54 ↓
203859 ncm-dbt-01 1216598 500 192 61 247 +93.2 ± 13.91 0 10 103 133 4 +194.57 ± 33.53 ↓
203858 ncm-dbt-05 1221536 500 209 52 239 +112.91 ± 13.39 0 8 79 161 2 +251.89 ± 38.63 ↓
203857 ncm-dbt-02 1198250 500 203 56 241 +105.25 ± 14.35 0 10 89 145 6 +221.9 ± 36.31 ↓
203856 ncm-dbt-04 1197630 500 195 54 251 +100.7 ± 14.32 0 10 95 139 6 +209.91 ± 35.06 ↓
203855 ncm-dbt-03 1244488 500 185 57 258 +90.96 ± 15.33 0 22 80 146 2 +192.71 ± 37.95 ↓
203854 ncm-dbt-06 1236297 500 196 57 247 +99.2 ± 13.83 0 12 88 149 1 +215.85 ± 36.55 ↓
203853 ncm-dbt-01 1206054 500 198 66 236 +93.95 ± 15.53 1 18 83 144 4 +198.34 ± 37.46 ↓
203852 ncm-dbt-05 1237188 500 190 64 246 +89.48 ± 14.15 0 14 98 136 2 +189.0 ± 34.56 ↓
203851 ncm-dbt-02 1195547 500 188 58 254 +92.45 ± 15.64 0 20 86 138 6 +189.0 ± 36.81 ↓
203850 ncm-dbt-03 1239809 500 188 63 249 +88.74 ± 15.28 0 20 89 137 4 +183.51 ± 36.22 ↓
203849 ncm-dbt-04 1216356 500 198 57 245 +100.7 ± 13.36 0 5 104 136 5 +211.87 ± 33.02 ↓
203848 ncm-dbt-06 1232845 500 206 56 238 +107.54 ± 14.68 0 12 82 150 6 +228.08 ± 37.89 ↓
203847 ncm-dbt-01 1214768 500 185 61 254 +88.0 ± 15.13 0 20 89 138 3 +183.51 ± 36.22 ↓
203846 ncm-dbt-05 1218508 500 197 72 231 +88.74 ± 14.28 0 15 97 136 2 +187.16 ± 34.76 ↓
203845 ncm-dbt-02 1191034 500 194 59 247 +96.19 ± 14.41 0 14 90 143 3 +204.07 ± 36.13 ↓
203844 ncm-dbt-01 1195092 500 208 54 238 +110.6 ± 14.99 0 12 80 150 8 +232.26 ± 38.36 ↓
203843 ncm-dbt-04 1205798 500 199 60 241 +99.2 ± 14.45 0 13 89 144 4 +209.91 ± 36.34 ↓
203842 ncm-dbt-03 1233661 500 201 51 248 +107.54 ± 13.89 0 8 89 148 5 +230.16 ± 36.26 ↓
203841 ncm-dbt-06 1253105 500 182 64 254 +83.57 ± 14.45 0 15 106 125 4 +171.02 ± 33.15 ↓
176047 ncm-dbt-02 1452487 6 2 1 3 +58.39 ± 217.67 0 1 0 2 0 +120.37 ± 762.49 ↓
176045 ncm-dbt-04 1419806 44 19 4 21 +123.36 ± 58.22 0 1 7 12 2 +235.93 ± 144.01 ↓
176044 ncm-dbt-02 1443551 50 17 5 28 +85.04 ± 41.74 0 1 11 13 0 +181.7 ± 107.11 ↓
176043 ncm-dbt-01 1440067 50 19 7 24 +85.03 ± 36.28 0 0 13 12 0 +181.68 ± 91.94 ↓
176042 ncm-dbt-05 1437463 50 18 7 25 +77.7 ± 41.12 0 1 12 12 0 +164.07 ± 100.82 ↓
176041 ncm-dbt-04 1424205 50 20 5 25 +107.52 ± 52.35 0 2 7 15 1 +219.87 ± 140.03 ↓
176040 ncm-dbt-02 1455885 50 16 4 30 +85.04 ± 46.59 0 2 9 14 0 +181.7 ± 121.14 ↓
176039 ncm-dbt-01 1441081 50 19 6 25 +92.44 ± 55.7 0 3 7 14 1 +181.7 ± 134.48 ↓
176038 ncm-dbt-05 1435011 50 20 8 22 +85.02 ± 51.0 0 1 13 9 2 +147.19 ± 95.14 ↓
176037 ncm-dbt-04 1423454 50 20 6 24 +99.93 ± 36.88 0 0 11 14 0 +219.84 ± 104.3 ↓
176036 ncm-dbt-02 1450139 50 16 6 28 +70.43 ± 40.35 0 1 13 11 0 +147.19 ± 95.14 ↓
176035 ncm-dbt-01 1439051 50 18 4 28 +99.94 ± 42.49 0 1 9 15 0 +219.87 ± 122.63 ↓
176033 ncm-dbt-05 1437905 50 21 5 24 +115.22 ± 42.56 0 1 7 17 0 +263.42 ± 145.5 ↓
176027 ncm-dbt-04 1419367 50 17 7 26 +70.43 ± 40.35 0 1 13 11 0 +147.19 ± 95.14 ↓
176026 ncm-dbt-01 1437751 50 20 11 19 +63.21 ± 56.42 0 3 12 8 2 +99.95 ± 101.68 ↓
176025 ncm-dbt-02 1445603 50 17 4 29 +92.44 ± 51.56 0 2 9 13 1 +181.7 ± 121.14 ↓
176024 ncm-dbt-05 1437894 50 24 6 20 +130.93 ± 41.82 0 1 5 19 0 +315.35 ± 187.35 ↓
176023 ncm-dbt-04 1426435 50 22 8 20 +99.93 ± 56.22 0 2 9 12 2 +181.7 ± 121.14 ↓
176022 ncm-dbt-01 1434166 50 17 8 25 +63.22 ± 44.25 0 2 12 11 0 +130.94 ± 101.68 ↓
176021 ncm-dbt-02 1445739 50 23 5 22 +130.92 ± 57.23 0 2 5 16 2 +263.42 ± 170.28 ↓
176020 ncm-dbt-05 1435453 50 18 4 28 +99.93 ± 47.48 0 1 10 13 1 +200.24 ± 114.26 ↓

Commit

Commit ID c079acc26f93acc2eda08c7218c60559854f52f0
Author Tomasz Sobczyk
Date 2022-05-14 10:47:22 UTC
Update NNUE architecture to SFNNv5. Update network to nn-3c0aa92af1da.nnue. Architecture changes: Duplicated activation after the 1024->15 layer with squared crelu (so 15->15*2). As proposed by vondele. Trainer changes: Added bias to L1 factorization, which was previously missing (no measurable improvement but at least neutral in principle) For retraining linearly reduce lambda parameter from 1.0 at epoch 0 to 0.75 at epoch 800. reduce max_skipping_rate from 15 to 10 (compared to vondele's outstanding PR) Note: This network was trained with a ~0.8% error in quantization regarding the newly added activation function. This will be fixed in the released trainer version. Expect a trainer PR tomorrow. Note: The inference implementation cuts a corner to merge results from two activation functions. This could possibly be resolved nicer in the future. AVX2 implementation likely not necessary, but NEON is missing. First training session invocation: python3 train.py \ ../nnue-pytorch-training/data/nodes5000pv2_UHO.binpack \ ../nnue-pytorch-training/data/nodes5000pv2_UHO.binpack \ --gpus "$3," \ --threads 4 \ --num-workers 8 \ --batch-size 16384 \ --progress_bar_refresh_rate 20 \ --random-fen-skipping 3 \ --features=HalfKAv2_hm^ \ --lambda=1.0 \ --max_epochs=400 \ --default_root_dir ../nnue-pytorch-training/experiment_$1/run_$2 Second training session invocation: python3 train.py \ ../nnue-pytorch-training/data/T60T70wIsRightFarseerT60T74T75T76.binpack \ ../nnue-pytorch-training/data/T60T70wIsRightFarseerT60T74T75T76.binpack \ --gpus "$3," \ --threads 4 \ --num-workers 8 \ --batch-size 16384 \ --progress_bar_refresh_rate 20 \ --random-fen-skipping 3 \ --features=HalfKAv2_hm^ \ --start-lambda=1.0 \ --end-lambda=0.75 \ --gamma=0.995 \ --lr=4.375e-4 \ --max_epochs=800 \ --resume-from-model /data/sopel/nnue/nnue-pytorch-training/data/exp367/nn-exp367-run3-epoch399.pt \ --default_root_dir ../nnue-pytorch-training/experiment_$1/run_$2 Passed STC: LLR: 2.95 (-2.94,2.94) <0.00,2.50> Total: 27288 W: 7445 L: 7178 D: 12665 Ptnml(0-2): 159, 3002, 7054, 3271, 158 https://tests.stockfishchess.org/tests/view/627e8c001919125939623644 Passed LTC: LLR: 2.95 (-2.94,2.94) <0.50,3.00> Total: 21792 W: 5969 L: 5727 D: 10096 Ptnml(0-2): 25, 2152, 6294, 2406, 19 https://tests.stockfishchess.org/tests/view/627f2a855734b18b2e2ece47 closes https://github.com/official-stockfish/Stockfish/pull/4020 Bench: 6481017
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version