Dev Builds » 20230209-0650

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 09:38:00 1953545 3352 2869 4 479 +442.42 ± 15.56
ncm-et-4 09:38:04 1952773 3358 2866 5 487 +438.95 ± 15.43
ncm-et-9 09:37:34 1949699 3322 2860 5 457 +448.58 ± 15.94
ncm-et-10 09:37:52 1946062 3326 2878 2 446 +455.73 ± 16.13
ncm-et-13 09:38:03 1954852 3323 2857 6 460 +446.65 ± 15.89
ncm-et-15 09:38:00 1951496 3319 2889 5 425 +461.65 ± 16.54
20000 17219 27 2754 +448.82 ± 6.48

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
179135 ncm-et-9 2023-03-16 22:43 00:55:35 1954504 322 280 1 41 +458.16 ± 54.58
179134 ncm-et-15 2023-03-16 22:43 00:56:13 1952521 319 285 0 34 +499.81 ± 60.28
179133 ncm-et-13 2023-03-16 22:42 00:56:31 1957863 323 273 2 48 +423.11 ± 50.21
179132 ncm-et-10 2023-03-16 22:42 00:56:23 1939489 326 287 0 39 +478.55 ± 55.97
179130 ncm-et-3 2023-03-16 22:39 00:59:52 1966333 352 299 0 53 +435.72 ± 47.53
179126 ncm-et-4 2023-03-16 22:37 01:01:49 1958188 358 308 0 50 +449.79 ± 49.04
179062 ncm-et-9 2023-03-16 21:16 01:26:19 1933506 500 423 1 76 +429.05 ± 39.52
179061 ncm-et-10 2023-03-16 21:15 01:26:32 1934648 500 425 0 75 +436.43 ± 39.73
179060 ncm-et-15 2023-03-16 21:15 01:27:05 1948726 500 449 1 50 +504.32 ± 49.24
179059 ncm-et-13 2023-03-16 21:14 01:27:39 1943384 500 428 0 72 +444.08 ± 40.59
179058 ncm-et-3 2023-03-16 21:12 01:26:02 1957153 500 436 0 64 +466.03 ± 43.18
179057 ncm-et-4 2023-03-16 21:11 01:25:03 1961543 500 427 0 73 +441.5 ± 40.29
178984 ncm-et-9 2023-03-16 19:48 01:27:30 1954422 500 428 0 72 +444.08 ± 40.59
178983 ncm-et-13 2023-03-16 19:48 01:25:42 1959709 500 435 1 64 +460.32 ± 43.24
178982 ncm-et-15 2023-03-16 19:47 01:26:56 1954655 500 427 2 71 +436.43 ± 40.99
178981 ncm-et-10 2023-03-16 19:47 01:27:21 1954020 500 437 0 63 +468.95 ± 43.54
178979 ncm-et-3 2023-03-16 19:45 01:26:35 1948755 500 422 1 77 +426.65 ± 39.25
178976 ncm-et-4 2023-03-16 19:43 01:27:44 1931173 500 418 1 81 +417.32 ± 38.22
178917 ncm-et-13 2023-03-16 18:21 01:26:10 1954518 500 433 1 66 +454.76 ± 42.55
178915 ncm-et-10 2023-03-16 18:20 01:26:34 1943023 500 435 0 65 +463.15 ± 42.83
178913 ncm-et-9 2023-03-16 18:19 01:28:12 1944876 500 427 0 73 +441.5 ± 40.29
178911 ncm-et-15 2023-03-16 18:18 01:28:18 1944973 500 435 1 64 +460.32 ± 43.24
178910 ncm-et-3 2023-03-16 18:18 01:26:12 1944320 500 426 1 73 +436.43 ± 40.36
178909 ncm-et-4 2023-03-16 18:18 01:24:25 1963075 500 427 2 71 +436.43 ± 40.99
178867 ncm-et-9 2023-03-16 16:53 01:24:55 1963095 500 435 2 63 +457.52 ± 43.63
178866 ncm-et-13 2023-03-16 16:53 01:26:45 1957441 500 425 1 74 +433.94 ± 40.08
178865 ncm-et-10 2023-03-16 16:53 01:26:12 1950704 500 441 1 58 +477.99 ± 45.54
178863 ncm-et-4 2023-03-16 16:52 01:24:51 1963260 500 428 0 72 +444.08 ± 40.59
178859 ncm-et-15 2023-03-16 16:51 01:26:07 1961111 500 427 0 73 +441.5 ± 40.29
178858 ncm-et-3 2023-03-16 16:50 01:27:52 1940104 500 427 1 72 +438.95 ± 40.66
178825 ncm-et-13 2023-03-16 15:25 01:27:53 1952123 500 436 1 63 +463.16 ± 43.6
178823 ncm-et-9 2023-03-16 15:25 01:28:00 1946025 500 438 1 61 +468.96 ± 44.35
178822 ncm-et-10 2023-03-16 15:25 01:27:39 1945598 500 422 0 78 +429.05 ± 38.92
178821 ncm-et-4 2023-03-16 15:25 01:27:07 1942992 500 434 1 65 +457.52 ± 42.89
178820 ncm-et-15 2023-03-16 15:24 01:26:29 1956058 500 430 0 70 +449.35 ± 41.19
178819 ncm-et-3 2023-03-16 15:23 01:26:22 1951927 500 432 0 68 +454.76 ± 41.82
178776 ncm-et-9 2023-03-16 13:57 01:27:03 1951468 500 429 0 71 +446.7 ± 40.89
178775 ncm-et-3 2023-03-16 13:57 01:25:05 1966223 500 427 1 72 +438.95 ± 40.66
178774 ncm-et-4 2023-03-16 13:57 01:27:05 1949186 500 424 1 75 +431.48 ± 39.8
178773 ncm-et-15 2023-03-16 13:57 01:26:52 1942434 500 436 1 63 +463.16 ± 43.6
178772 ncm-et-13 2023-03-16 13:57 01:27:23 1958929 500 427 0 73 +441.5 ± 40.29
178771 ncm-et-10 2023-03-16 13:57 01:27:11 1954958 500 431 1 68 +449.35 ± 41.89

Commit

Commit ID 05dea2ca4657dec10637bb53c4ad583f680e0677
Author Linmiao Xu
Date 2023-02-09 06:50:27 UTC
Update default net to nn-1337b1adec5b.nnue Created by retraining the master net on a dataset composed of: * Most of the previous best dataset filtered to remove positions likely having only one good move * Adding training data from Leela T77 dec2021 rescored with 16tb of 7-piece tablebases Trained with end lambda 0.7 and max epoch 900. Positions with ply <= 28 were removed from most of the previous best dataset before training began. A new nnue-pytorch trainer param for skipping early plies was used to skip plies <= 24 in the unfiltered and additional Leela T77 parts of the dataset. ``` python easy_train.py \ --experiment-name leela96-dfrc99-T80octnovT79aprmayT60novdec-eval-filt-v2-T78augsep-12tb-T77dec-16tb-lambda7-sk24 \ --training-dataset /data/leela96-dfrc99-T80octnovT79aprmayT60novdec-eval-filt-v2-T78augsep-12tb-T77dec-16tb.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/easy-train-early-fen-skipping \ --early-fen-skipping 24 \ --gpus "0," \ --start-from-engine-test-net True \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 900 ``` The depth6 multipv2 search filtering method is the same as the one used for filtering recent best datasets, with a lower eval difference threshold to remove slightly more positions than before. These parts of the dataset were filtered: * 96% of T60T70wIsRightFarseerT60T74T75T76.binpack * 99% of dfrc_n5000.binpack * T80 oct + nov 2022 data, no positions with castling flags, rescored with ~600gb 7p tablebases * T79 apr + may 2022 data, rescored with 12tb 7p tablebases * T60 nov + dec 2021 data, rescored with 12tb 7p tablebases These parts of the dataset were not filtered. Positions with ply <= 24 were skipped during training: * T78 aug + sep 2022 data, rescored with 12tb 7p tablebases * 84% of T77 dec 2021 data, rescored with 16tb 7p tablebases The code and exact evaluation thresholds used for data filtering can be found at: https://github.com/linrock/Stockfish/tree/tools-filter-multipv2-eval-diff-t2/src/filter The exact training data used can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch859.nnue : 3.5 +/ 1.2 Passed STC: LLR: 2.95 (-2.94,2.94) <0.00,2.00> https://tests.stockfishchess.org/tests/view/63dfeefc73223e7f52ad769f Total: 219744 W: 58572 L: 58002 D: 103170 Ptnml(0-2): 609, 24446, 59284, 24832, 701 Passed LTC: https://tests.stockfishchess.org/tests/view/63e268fc73223e7f52ade7b6 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 91256 W: 24528 L: 24121 D: 42607 Ptnml(0-2): 48, 8863, 27390, 9288, 39 closes https://github.com/official-stockfish/Stockfish/pull/4387 bench 3841998
Copyright 2011–2024 Next Chess Move LLC