Dev Builds » 20230329-1937

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 10:45:30 1212065 3604 1496 339 1769 +115.63 ± 5.1 1 49 578 1140 34 +254.02 ± 14.16
ncm-dbt-02 10:40:37 1225742 3564 1452 335 1777 +112.68 ± 5.25 1 63 569 1116 33 +245.68 ± 14.29
ncm-dbt-03 10:44:19 1232199 3562 1482 338 1742 +115.68 ± 5.22 1 53 567 1121 39 +252.46 ± 14.31
ncm-dbt-04 10:45:39 1239630 3604 1485 332 1787 +115.2 ± 5.26 1 60 567 1133 41 +250.57 ± 14.31
ncm-dbt-05 06:20:17 1225930 2098 864 194 1040 +114.97 ± 6.97 0 44 312 672 21 +251.15 ± 19.31
ncm-dbt-06 10:45:15 1221939 3568 1455 345 1768 +111.79 ± 5.09 0 49 607 1097 31 +243.45 ± 13.8
20000 8234 1883 9883 +114.28 ± 2.2 4 318 3200 6279 199 +249.42 ± 6.02

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
181650 ncm-dbt-03 1237261 62 21 2 39 +109.99 ± 33.02 0 0 12 19 0 +247.89 ± 101.21
181649 ncm-dbt-02 1238689 64 28 10 26 +100.42 ± 40.04 0 2 10 20 0 +221.14 ± 115.01
181648 ncm-dbt-06 1216295 68 29 9 30 +105.28 ± 31.59 0 0 14 20 0 +234.48 ± 92.12
181647 ncm-dbt-04 1230559 104 46 8 50 +133.09 ± 32.4 0 2 12 36 2 +296.15 ± 104.93
181646 ncm-dbt-05 1231469 98 39 11 48 +102.1 ± 30.37 0 2 17 30 0 +225.71 ± 85.9
181645 ncm-dbt-01 1236290 104 43 8 53 +121.66 ± 29.11 0 2 13 37 0 +283.63 ± 100.29
181638 ncm-dbt-02 1229844 500 204 56 240 +106.01 ± 14.04 0 11 83 153 3 +230.16 ± 37.67
181637 ncm-dbt-06 1220401 500 206 49 245 +112.91 ± 13.56 0 6 86 153 5 +245.2 ± 36.85
181636 ncm-dbt-03 1235672 500 211 48 241 +117.55 ± 14.37 0 9 76 158 7 +254.16 ± 39.42
181635 ncm-dbt-01 1200755 500 203 40 257 +117.55 ± 14.04 0 12 65 171 2 +265.78 ± 42.44
181634 ncm-dbt-04 1246398 500 217 46 237 +123.81 ± 14.48 0 7 75 158 10 +265.78 ± 39.69
181633 ncm-dbt-05 1224333 500 208 45 247 +117.55 ± 14.53 0 11 71 162 6 +256.44 ± 40.73
181626 ncm-dbt-02 1233274 500 198 40 262 +113.68 ± 13.72 0 6 86 152 6 +245.2 ± 36.85
181625 ncm-dbt-06 1213483 500 196 48 256 +106.01 ± 14.2 0 10 87 148 5 +226.0 ± 36.75
181624 ncm-dbt-03 1249216 500 205 50 245 +111.37 ± 13.22 0 5 89 152 4 +243.0 ± 36.1
181623 ncm-dbt-01 1229968 500 206 46 248 +115.22 ± 13.72 0 7 81 157 5 +251.89 ± 38.11
181622 ncm-dbt-04 1233812 500 199 45 256 +110.6 ± 13.89 0 7 88 149 6 +236.51 ± 36.44
181621 ncm-dbt-05 1240752 500 209 46 245 +117.55 ± 14.04 0 10 71 165 4 +261.07 ± 40.78
181614 ncm-dbt-02 1222424 500 199 47 254 +109.07 ± 14.21 0 12 77 158 3 +238.66 ± 39.1
181613 ncm-dbt-06 1220603 500 202 38 260 +118.33 ± 13.0 0 4 82 160 4 +263.42 ± 37.69
181612 ncm-dbt-03 1235772 500 220 46 234 +126.17 ± 13.95 0 7 69 167 7 +280.42 ± 41.44
181611 ncm-dbt-01 1234832 500 207 52 241 +111.37 ± 13.73 0 7 86 152 5 +240.82 ± 36.9
181610 ncm-dbt-04 1237233 500 211 42 247 +122.24 ± 14.66 0 10 69 163 8 +265.78 ± 41.36
181609 ncm-dbt-05 1212115 500 201 50 249 +108.3 ± 14.52 0 11 83 150 6 +230.16 ± 37.67
181602 ncm-dbt-02 1223398 500 200 56 244 +102.97 ± 14.94 1 11 87 145 6 +217.85 ± 36.77
181601 ncm-dbt-06 1228859 500 210 58 232 +109.07 ± 14.05 0 8 88 148 6 +232.26 ± 36.48
181600 ncm-dbt-01 1192789 500 218 54 228 +118.33 ± 13.52 1 3 82 159 5 +263.42 ± 37.69
181599 ncm-dbt-03 1219433 500 199 50 251 +106.78 ± 14.2 1 7 89 148 5 +230.16 ± 36.26
181598 ncm-dbt-04 1235716 500 198 45 257 +109.83 ± 14.06 0 11 78 158 3 +240.82 ± 38.87
181597 ncm-dbt-05 1220982 500 207 42 251 +119.11 ± 14.19 0 10 70 165 5 +263.42 ± 41.06
181590 ncm-dbt-02 1233031 500 204 36 260 +121.45 ± 13.14 0 6 73 168 3 +275.45 ± 40.24
181589 ncm-dbt-06 1215366 500 208 58 234 +107.54 ± 13.56 0 9 84 155 2 +236.51 ± 37.42
181588 ncm-dbt-03 1222883 500 206 59 235 +105.25 ± 14.35 0 12 83 151 4 +226.0 ± 37.66
181587 ncm-dbt-01 1188654 500 201 43 256 +113.68 ± 13.72 0 7 83 155 5 +247.41 ± 37.61
181586 ncm-dbt-04 1239749 500 192 44 264 +106.01 ± 13.05 0 8 86 156 0 +236.51 ± 36.93
181579 ncm-dbt-02 1215845 500 207 51 242 +112.14 ± 14.38 0 8 86 148 8 +236.51 ± 36.93
181578 ncm-dbt-06 1229523 500 212 45 243 +120.67 ± 13.84 0 8 72 165 5 +268.17 ± 40.54
181577 ncm-dbt-01 1191466 500 206 50 244 +112.14 ± 14.06 0 8 84 152 6 +240.82 ± 37.4
181576 ncm-dbt-03 1226803 500 213 35 252 +129.35 ± 12.8 0 3 71 171 5 +295.94 ± 40.69
181575 ncm-dbt-04 1255529 500 210 56 234 +110.6 ± 14.69 0 11 81 151 7 +234.38 ± 38.14
181568 ncm-dbt-03 1230555 500 207 48 245 +114.45 ± 14.53 0 10 78 155 7 +245.2 ± 38.89
181567 ncm-dbt-02 1209431 500 212 39 249 +125.38 ± 13.43 0 7 67 172 4 +285.49 ± 42.08
181566 ncm-dbt-04 1238047 500 212 46 242 +119.89 ± 13.68 1 4 78 162 5 +268.17 ± 38.8
181565 ncm-dbt-01 1221770 500 212 46 242 +119.89 ± 13.16 0 3 84 157 6 +263.42 ± 37.11
181564 ncm-dbt-06 1230985 500 192 40 268 +109.07 ± 13.05 0 4 94 148 4 +236.51 ± 34.92

Commit

Commit ID 37160c4b1632245d46d86cec7bd22b76f5a87531
Author Linmiao Xu
Date 2023-03-29 19:37:52 UTC
Update default net to nn-dabb1ed23026.nnue Created by retraining the master net with these modifications: * New filtering methods for existing data from T80 sep+oct2022, T79 apr2022, T78 jun+jul+aug+sep2022, T77 dec2021 * Adding new filtered data from T80 aug2022 and T78 apr+may2022 * Increasing early-fen-skipping from 28 to 30 ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3-sk30 \ --training-dataset /data/leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --max_epoch 900 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The v3 filtering used for data from T77dec 2021 differs from v2 filtering in that: * To improve binpack compression, positions after ply 28 were skipped during training by setting position scores to VALUE_NONE (32002) instead of removing them entirely * All early-game positions with ply <= 28 were removed to maximize binpack compression * Only bestmove captures at d6pv2 search were skipped, not 2nd bestmove captures * Binpack compression was repaired for the remaining positions by effectively replacing bestmoves with "played moves" to maintain contiguous sequences of positions in the training game data After improving binpack compression, The T77 dec2021 data size was reduced from 95G to 19G. The v6 filtering used for data from T80augsepoctT79aprT78aprtosep 2022 differs from v2 in that: * All positions with only one legal move were removed * Tighter score differences at d6pv2 search were used to remove more positions with only one good move than before * d6pv2 search was not used to remove positions where the best 2 moves were captures ``` python3 interleave_binpacks.py \ nn-547-dataset/leela96-eval-filt-v2.binpack \ nn-547-dataset/dfrc99-eval-filt-v2.binpack \ nn-547-dataset/test80-nov2022-12tb7p-eval-filt-v2-d6.binpack \ nn-547-dataset/T79-may2022-12tb7p-eval-filt-v2.binpack \ nn-547-dataset/T60-nov2021-12tb7p-eval-filt-v2.binpack \ nn-547-dataset/T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.binpack \ filt-v6/test80-oct2022-16tb7p-filter-v6.binpack \ filt-v6/test79-apr2022-16tb7p-filter-v6.binpack \ filt-v6/test78-aprmay2022-16tb7p-filter-v6.binpack \ filt-v6/test78-junjulaug2022-16tb7p-filter-v6.binpack \ filt-v6/test78-sep2022-16tb7p-filter-v6.binpack \ filt-v3/test77-dec2021-16tb7p-filt-v3.binpack \ /data/leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3.binpack ``` The code for the new data filtering methods is available at: https://github.com/linrock/Stockfish/tree/nnue-data-v3/nnue-data The code for giving hexword names to .nnue files is at: https://github.com/linrock/nnue-namer Links for downloading the training data components can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch779.nnue : 0.6 +/- 3.1 Passed STC: https://tests.stockfishchess.org/tests/view/64212412db43ab2ba6f8efb0 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 82256 W: 22185 L: 21809 D: 38262 Ptnml(0-2): 286, 9065, 22067, 9407, 303 Passed LTC: https://tests.stockfishchess.org/tests/view/64223726db43ab2ba6f91d6c LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 30840 W: 8437 L: 8149 D: 14254 Ptnml(0-2): 14, 2891, 9323, 3177, 15 closes https://github.com/official-stockfish/Stockfish/pull/4465 bench 5101970
Copyright 2011–2024 Next Chess Move LLC