Dev Builds » 20230123-0601

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 11:13:09 1954590 3327 2833 8 486 +435.33 ± 15.46
ncm-et-4 11:13:13 1962328 3370 2905 2 463 +451.26 ± 15.82
ncm-et-9 11:13:00 1956456 3334 2885 6 443 +454.12 ± 16.2
ncm-et-10 11:13:07 1942870 3304 2844 2 458 +449.58 ± 15.91
ncm-et-13 11:12:54 1956547 3323 2891 4 428 +461.44 ± 16.48
ncm-et-15 11:13:16 1957245 3342 2883 4 455 +451.31 ± 15.97
20000 17241 26 2733 +450.36 ± 6.51

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
164166 ncm-et-10 2023-01-23 21:13 01:02:38 1941057 304 257 0 47 +430.74 ± 50.59
164165 ncm-et-13 2023-01-23 21:10 01:05:30 1957096 323 283 0 40 +472.16 ± 55.21
164164 ncm-et-3 2023-01-23 21:09 01:07:08 1936107 327 274 0 53 +421.83 ± 47.49
164163 ncm-et-9 2023-01-23 21:08 01:08:05 1954527 334 300 0 34 +508.23 ± 60.3
164162 ncm-et-15 2023-01-23 21:06 01:09:34 1952672 342 287 0 55 +423.31 ± 46.59
164161 ncm-et-4 2023-01-23 21:03 01:12:44 1968949 370 319 0 51 +452.25 ± 48.54
164160 ncm-et-10 2023-01-23 19:30 01:42:16 1956635 500 439 0 61 +474.93 ± 44.28
164159 ncm-et-13 2023-01-23 19:29 01:40:25 1970061 500 431 1 68 +449.35 ± 41.89
164158 ncm-et-3 2023-01-23 19:28 01:40:18 1965228 500 426 1 73 +436.43 ± 40.36
164157 ncm-et-9 2023-01-23 19:25 01:42:18 1947961 500 427 1 72 +438.95 ± 40.66
164156 ncm-et-15 2023-01-23 19:24 01:41:57 1953281 500 436 0 64 +466.03 ± 43.18
164155 ncm-et-4 2023-01-23 19:23 01:39:23 1967148 500 424 0 76 +433.94 ± 39.45
164154 ncm-et-13 2023-01-23 17:47 01:41:43 1950361 500 429 0 71 +446.7 ± 40.89
164153 ncm-et-10 2023-01-23 17:47 01:42:46 1937341 500 420 1 79 +421.93 ± 38.73
164152 ncm-et-3 2023-01-23 17:45 01:42:20 1952064 500 421 1 78 +424.28 ± 38.99
164151 ncm-et-15 2023-01-23 17:44 01:39:14 1962611 500 421 1 78 +424.28 ± 38.99
164150 ncm-et-4 2023-01-23 17:43 01:39:54 1964933 500 432 1 67 +452.04 ± 42.22
164149 ncm-et-9 2023-01-23 17:41 01:42:35 1954555 500 442 2 56 +477.99 ± 46.39
164148 ncm-et-13 2023-01-23 16:05 01:40:52 1957216 500 436 1 63 +463.16 ± 43.6
164147 ncm-et-10 2023-01-23 16:04 01:42:31 1939832 500 421 0 79 +426.65 ± 38.66
164146 ncm-et-15 2023-01-23 16:04 01:39:32 1961151 500 438 0 62 +471.92 ± 43.9
164145 ncm-et-4 2023-01-23 16:03 01:39:10 1961735 500 421 0 79 +426.65 ± 38.66
164144 ncm-et-3 2023-01-23 16:02 01:41:36 1958799 500 428 2 70 +438.95 ± 41.29
164143 ncm-et-9 2023-01-23 16:01 01:39:14 1971414 500 430 0 70 +449.35 ± 41.19
164142 ncm-et-13 2023-01-23 14:23 01:41:46 1953756 500 440 0 60 +477.98 ± 44.67
164141 ncm-et-15 2023-01-23 14:22 01:40:43 1956485 500 425 1 74 +433.94 ± 40.08
164140 ncm-et-10 2023-01-23 14:21 01:41:35 1941798 500 433 1 66 +454.76 ± 42.55
164139 ncm-et-3 2023-01-23 14:21 01:40:24 1955732 500 426 1 73 +436.43 ± 40.36
164138 ncm-et-4 2023-01-23 14:21 01:41:13 1948593 500 431 0 69 +452.04 ± 41.5
164137 ncm-et-9 2023-01-23 14:19 01:41:34 1952069 500 419 3 78 +415.05 ± 39.05
164136 ncm-et-9 2023-01-23 12:41 01:37:28 1960318 500 434 0 66 +460.32 ± 42.48
164135 ncm-et-4 2023-01-23 12:41 01:39:04 1966017 500 443 1 56 +484.25 ± 46.39
164134 ncm-et-3 2023-01-23 12:41 01:39:42 1959244 500 426 2 72 +433.94 ± 40.69
164133 ncm-et-15 2023-01-23 12:41 01:40:35 1956350 500 433 0 67 +457.52 ± 42.15
164132 ncm-et-13 2023-01-23 12:40 01:41:59 1956335 500 435 2 63 +457.52 ± 43.63
164131 ncm-et-10 2023-01-23 12:38 01:42:20 1923105 500 434 0 66 +460.32 ± 42.48
164130 ncm-et-13 2023-01-23 10:59 01:40:39 1951006 500 437 0 63 +468.95 ± 43.54
164129 ncm-et-9 2023-01-23 10:59 01:41:46 1954353 500 433 0 67 +457.52 ± 42.15
164128 ncm-et-4 2023-01-23 10:59 01:41:45 1958927 500 435 0 65 +463.15 ± 42.83
164127 ncm-et-10 2023-01-23 10:59 01:39:01 1960323 500 440 0 60 +477.98 ± 44.67
164126 ncm-et-3 2023-01-23 10:59 01:41:41 1954960 500 432 1 67 +452.04 ± 42.22
164125 ncm-et-15 2023-01-23 10:59 01:41:41 1958167 500 443 2 55 +481.1 ± 46.83

Commit

Commit ID 596a528c6a9ace6fb1a8407c86d972d96653418d
Author Linmiao Xu
Date 2023-01-23 06:01:32 UTC
Update default net to nn-bc24c101ada0.nnue Created by retraining the master net with Leela T78 data from Aug+Sep 2022 added to the previous best dataset. Trained with end lambda 0.7 and started with max epoch 800. All positions with ply <= 28 were skipped: ``` python easy_train.py \ --experiment-name leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p-sk28-lambda7 \ --training-dataset /data/leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-skip-ply-lteq-28 \ --start-from-engine-test-net True \ --gpus "0," \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 800 ``` Around epoch 750, training was manually paused and max epoch increased to 950 before resuming. The additional Leela training data from T78 was prepared in the same way as the previous best dataset. The exact training data used can be found at: https://robotmoon.com/nnue-training-data/ While the local elo ratings during this experiment were much lower than in recent master nets, several later epochs had a consistent elo above zero, and this was hypothesized to represent potential strength at slower time controls. Local elo at 25k nodes per move leela95-dfrc96-filt-only-T80octnov-T60novdecT78augsepT79aprmay-12tb7p-sk28-lambda7 nn-epoch819.nnue : 0.4 +/- 1.1 (nn-bc24c101ada0.nnue) nn-epoch799.nnue : 0.3 +/- 1.2 nn-epoch759.nnue : 0.3 +/- 1.1 nn-epoch839.nnue : 0.2 +/- 1.4 Passed STC https://tests.stockfishchess.org/tests/view/63cabf6f0eefe8694a0c6013 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 41608 W: 11161 L: 10848 D: 19599 Ptnml(0-2): 116, 4496, 11281, 4781, 130 Passed LTC https://tests.stockfishchess.org/tests/view/63cb1856344bb01c191af263 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 76760 W: 20517 L: 20137 D: 36106 Ptnml(0-2): 34, 7435, 23070, 7799, 42 closes https://github.com/official-stockfish/Stockfish/pull/4351 bench 3941848
Copyright 2011–2024 Next Chess Move LLC