Dev Builds » 20240107-2020

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:48:34 1208042 3332 1451 223 1658 +134.37 ± 5.21 0 34 417 1168 47 +307.46 ± 16.71
ncm-dbt-02 09:44:58 1239742 3312 1470 251 1591 +134.17 ± 5.08 0 27 423 1166 40 +309.61 ± 16.58
ncm-dbt-03 09:43:45 1242445 3342 1474 250 1618 +133.44 ± 5.09 0 29 430 1171 41 +306.81 ± 16.44
ncm-dbt-04 09:49:18 1231240 3338 1461 242 1635 +133.02 ± 5.2 0 35 425 1164 45 +303.67 ± 16.55
ncm-dbt-05 09:47:23 1229362 3324 1485 241 1598 +136.66 ± 5.26 0 31 412 1163 56 +311.63 ± 16.81
ncm-dbt-06 09:47:45 1223689 3352 1470 242 1640 +133.48 ± 5.19 0 33 429 1167 47 +304.52 ± 16.47
20000 8811 1449 9740 +134.19 ± 2.11 0 189 2536 6999 276 +307.26 ± 6.76

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
244336 ncm-dbt-02 1221744 312 134 22 156 +130.53 ± 15.78 0 2 42 110 2 +304.85 ± 53.44
244335 ncm-dbt-05 1241934 324 146 21 157 +141.35 ± 17.92 0 5 34 116 7 +321.48 ± 59.54
244334 ncm-dbt-01 1228394 332 144 20 168 +136.36 ± 15.71 0 3 39 121 3 +321.87 ± 55.74
244333 ncm-dbt-03 1246532 342 149 31 162 +125.0 ± 16.09 0 4 48 116 3 +283.27 ± 49.93
244332 ncm-dbt-04 1264244 338 148 27 163 +130.14 ± 16.09 0 3 46 116 4 +296.15 ± 51.03
244331 ncm-dbt-06 1227615 352 151 25 176 +130.13 ± 14.7 0 4 42 130 0 +312.41 ± 53.58
244330 ncm-dbt-02 1222866 500 227 42 231 +134.95 ± 12.84 0 3 65 176 6 +312.48 ± 42.67
244329 ncm-dbt-05 1221604 500 220 38 242 +132.54 ± 13.83 0 7 61 175 7 +301.33 ± 44.13
244328 ncm-dbt-06 1221883 500 215 41 244 +126.17 ± 13.77 0 4 77 160 9 +275.45 ± 39.01
244327 ncm-dbt-04 1216322 500 221 41 238 +130.94 ± 13.14 0 4 68 172 6 +298.62 ± 41.71
244326 ncm-dbt-01 1230219 500 225 34 241 +139.81 ± 12.87 0 2 63 177 8 +324.17 ± 43.32
244325 ncm-dbt-03 1249195 500 216 32 252 +134.15 ± 13.43 0 4 66 172 8 +304.07 ± 42.38
244324 ncm-dbt-02 1248128 500 221 45 234 +127.76 ± 14.1 0 7 68 167 8 +282.94 ± 41.76
244323 ncm-dbt-05 1235983 500 227 43 230 +134.15 ± 13.43 0 4 66 172 8 +304.07 ± 42.38
244322 ncm-dbt-01 1206515 500 204 40 256 +118.33 ± 13.35 0 7 75 165 3 +265.78 ± 39.69
244321 ncm-dbt-04 1237455 500 224 27 249 +144.71 ± 12.68 0 3 54 186 7 +346.12 ± 47.1
244320 ncm-dbt-03 1249509 500 210 31 259 +130.14 ± 12.39 0 5 62 182 1 +309.64 ± 43.82
244319 ncm-dbt-06 1221697 500 223 36 241 +136.56 ± 13.73 0 8 53 183 6 +318.25 ± 47.23
244234 ncm-dbt-05 1221132 500 218 34 248 +134.15 ± 13.61 0 6 61 176 7 +306.84 ± 44.17
244233 ncm-dbt-02 1245261 500 227 42 231 +134.95 ± 13.22 0 4 64 175 7 +309.64 ± 43.08
244232 ncm-dbt-04 1234616 500 212 34 254 +129.35 ± 13.17 0 6 64 176 4 +298.62 ± 43.1
244231 ncm-dbt-01 1187647 500 220 31 249 +138.18 ± 13.69 0 6 57 179 8 +318.25 ± 45.73
244230 ncm-dbt-06 1217461 500 214 35 251 +130.14 ± 11.99 0 2 69 177 2 +306.84 ± 41.23
244229 ncm-dbt-03 1248516 500 220 35 245 +134.95 ± 12.64 0 2 67 175 6 +312.48 ± 41.9
244228 ncm-dbt-05 1231764 500 221 40 239 +131.74 ± 13.67 0 4 70 167 9 +293.29 ± 41.07
244227 ncm-dbt-02 1237787 500 233 28 239 +151.34 ± 11.91 0 2 47 195 6 +377.87 ± 50.67
244226 ncm-dbt-04 1229285 500 213 43 244 +123.02 ± 14.16 0 8 71 164 7 +270.57 ± 40.83
244225 ncm-dbt-06 1201927 500 213 40 247 +125.38 ± 13.79 0 7 69 168 6 +280.42 ± 41.44
244224 ncm-dbt-01 1223967 500 215 31 254 +134.15 ± 13.61 0 6 61 176 7 +306.84 ± 44.17
244223 ncm-dbt-03 1244791 500 233 38 229 +143.07 ± 13.73 0 5 55 180 10 +330.23 ± 46.62
244222 ncm-dbt-04 1234580 500 219 36 245 +133.34 ± 13.81 0 6 63 173 8 +301.33 ± 43.45
244221 ncm-dbt-05 1219579 500 231 35 234 +143.89 ± 12.51 0 1 60 181 8 +339.63 ± 44.35
244220 ncm-dbt-02 1241668 500 210 36 254 +126.17 ± 13.6 0 5 73 165 7 +280.42 ± 40.21
244219 ncm-dbt-01 1196468 500 220 34 246 +135.76 ± 13.76 0 5 63 173 9 +306.84 ± 43.45
244218 ncm-dbt-06 1239387 500 230 26 244 +150.51 ± 13.64 0 2 56 178 14 +346.12 ± 46.15
244217 ncm-dbt-03 1228498 500 227 35 238 +140.62 ± 13.24 0 3 61 177 9 +324.17 ± 44.15
244216 ncm-dbt-04 1202179 500 224 34 242 +138.99 ± 13.67 0 5 59 177 9 +318.25 ± 44.96
244215 ncm-dbt-05 1233539 500 222 30 248 +140.62 ± 13.62 0 4 60 176 10 +321.19 ± 44.57
244214 ncm-dbt-02 1260743 500 218 36 246 +132.54 ± 12.71 0 4 64 178 4 +309.64 ± 43.08
244213 ncm-dbt-03 1230075 500 219 48 233 +123.81 ± 13.28 0 6 71 169 4 +280.42 ± 40.83
244212 ncm-dbt-01 1183089 500 223 33 244 +138.99 ± 13.67 0 5 59 177 9 +318.25 ± 44.96
244211 ncm-dbt-06 1235858 500 224 39 237 +134.95 ± 14.13 0 6 63 171 10 +301.33 ± 43.45

Commit

Commit ID f09adaa4a4c3cbb44e1ca8cc687a08dc3d58076e
Author Linmiao Xu
Date 2024-01-07 20:20:15 UTC
Update smallnet to nn-baff1ede1f90.nnue with wider eval range Created by training an L1-128 net from scratch with a wider range of evals in the training data and wld-fen-skipping disabled during training. The differences in this training data compared to the first dual nnue PR are: - removal of all positions with 3 pieces - when piece count >= 16, keep positions with simple eval above 750 - when piece count < 16, remove positions with simple eval above 3000 The asymmetric data filtering was meant to flatten the training data piece count distribution, which was previously heavily skewed towards positions with low piece counts. Additionally, the simple eval range where the smallnet is used was widened to cover more positions previously evaluated by the big net and simple eval. ```yaml experiment-name: 128--S1-hse-S7-v4-S3-v1-no-wld-skip training-dataset: - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-v4.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-v4.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-v4.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-v4.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-v4.binpack wld-fen-skipping: False start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 start-lambda: 1.0 end-lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: https://github.com/official-stockfish/nnue-pytorch/pull/259 FT weights permuted with 10k positions from fishpack32.binpack with: https://github.com/official-stockfish/nnue-pytorch/pull/254 Data filtered for high simple eval positions (v4) with: https://github.com/linrock/Stockfish/blob/b9c8440/src/tools/transform.cpp#L640-L675 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch319.nnue : -241.7 +/- 3.2 Passed STC vs. 36db936: https://tests.stockfishchess.org/tests/view/6576b3484d789acf40aabbfe LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 21920 W: 5680 L: 5381 D: 10859 Ptnml(0-2): 82, 2488, 5520, 2789, 81 Passed LTC vs. DualNNUE #4915: https://tests.stockfishchess.org/tests/view/65775c034d789acf40aac7e3 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 147606 W: 36619 L: 36063 D: 74924 Ptnml(0-2): 98, 16591, 39891, 17103, 120 closes https://github.com/official-stockfish/Stockfish/pull/4919 Bench: 1438336
Copyright 2011–2024 Next Chess Move LLC