Dev Builds » 20230611-1323

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 09:47:40 1907476 3299 2842 2 455 +450.51 +/- 15.96
ncm-et-4 09:46:48 1908206 3325 2844 3 478 +442.06 +/- 15.57
ncm-et-9 09:43:41 1950626 3410 2910 4 496 +439.2 +/- 15.29
ncm-et-10 09:46:34 1902378 3335 2877 2 456 +452.13 +/- 15.95
ncm-et-13 09:46:43 1904674 3310 2851 3 456 +449.92 +/- 15.95
ncm-et-15 09:46:46 1909464 3321 2849 11 461 +442.23 +/- 15.89
20000 17173 25 2802 +445.91 +/- 6.42

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
191659 ncm-et-3 2023-06-12 02:27 00:41:01 1952761 241 198 0 43 +403.59 +/- 52.88
191658 ncm-et-13 2023-06-12 02:25 00:43:34 1958994 252 221 1 30 +467.52 +/- 64.5
191657 ncm-et-15 2023-06-12 02:23 00:44:49 1953291 263 227 0 36 +453.55 +/- 58.31
191656 ncm-et-4 2023-06-12 02:23 00:45:14 1963093 269 232 0 37 +452.65 +/- 57.46
191655 ncm-et-10 2023-06-12 02:19 00:48:26 1950011 281 240 0 41 +441.61 +/- 54.39
191654 ncm-et-9 2023-06-12 02:18 00:50:28 1945549 300 266 0 34 +488.52 +/- 60.24
191647 ncm-et-3 2023-06-12 01:00 01:26:12 1943760 500 437 1 62 +466.04 +/- 43.97
191646 ncm-et-13 2023-06-12 00:59 01:25:13 1954649 500 436 0 64 +466.03 +/- 43.18
191645 ncm-et-15 2023-06-12 00:57 01:25:18 1955262 500 442 2 56 +477.99 +/- 46.39
191644 ncm-et-4 2023-06-12 00:56 01:25:48 1958177 500 422 0 78 +429.05 +/- 38.92
191643 ncm-et-10 2023-06-12 00:53 01:25:47 1940438 500 439 0 61 +474.93 +/- 44.28
191642 ncm-et-9 2023-06-12 00:51 01:26:06 1953595 500 427 0 73 +441.5 +/- 40.29
191635 ncm-et-3 2023-06-11 23:34 01:25:45 1955568 500 434 0 66 +460.32 +/- 42.48
191634 ncm-et-13 2023-06-11 23:33 01:25:05 1941657 500 432 0 68 +454.76 +/- 41.82
191633 ncm-et-4 2023-06-11 23:31 01:24:22 1961539 500 432 0 68 +454.76 +/- 41.82
191632 ncm-et-15 2023-06-11 23:30 01:27:02 1953834 500 433 1 66 +454.76 +/- 42.55
191631 ncm-et-10 2023-06-11 23:27 01:25:25 1944063 500 424 0 76 +433.94 +/- 39.45
191630 ncm-et-9 2023-06-11 23:25 01:25:32 1942077 500 426 0 74 +438.95 +/- 40.01
191623 ncm-et-3 2023-06-11 22:08 01:24:51 1948012 500 436 0 64 +466.03 +/- 43.18
191622 ncm-et-13 2023-06-11 22:07 01:25:30 1951027 500 438 0 62 +471.92 +/- 43.9
191621 ncm-et-4 2023-06-11 22:05 01:25:45 1949596 500 425 1 74 +433.94 +/- 40.08
191620 ncm-et-15 2023-06-11 22:05 01:24:00 1954240 500 428 4 68 +433.94 +/- 41.9
191619 ncm-et-10 2023-06-11 22:02 01:23:56 1960922 500 430 2 68 +444.09 +/- 41.92
191618 ncm-et-9 2023-06-11 21:57 01:26:31 1952978 500 425 0 75 +436.43 +/- 39.73
191611 ncm-et-3 2023-06-11 20:41 01:26:58 1959109 500 429 0 71 +446.7 +/- 40.89
191610 ncm-et-13 2023-06-11 20:40 01:26:20 1949689 500 434 0 66 +460.32 +/- 42.48
191609 ncm-et-15 2023-06-11 20:39 01:24:59 1947114 500 422 2 76 +424.28 +/- 39.56
191608 ncm-et-4 2023-06-11 20:39 01:25:29 1954043 500 433 1 66 +454.76 +/- 42.55
191607 ncm-et-10 2023-06-11 20:38 01:23:52 1948020 500 432 0 68 +454.76 +/- 41.82
191606 ncm-et-9 2023-06-11 20:33 01:23:59 1960328 500 423 3 74 +424.28 +/- 40.13
191599 ncm-et-4 2023-06-11 19:15 01:23:18 1964515 500 423 1 76 +429.05 +/- 39.52
191598 ncm-et-15 2023-06-11 19:13 01:25:31 1953314 500 422 2 76 +424.28 +/- 39.56
191597 ncm-et-3 2023-06-11 19:13 01:26:54 1947352 500 426 1 73 +436.43 +/- 40.36
191596 ncm-et-13 2023-06-11 19:13 01:26:13 1950409 500 419 1 80 +419.61 +/- 38.47
191595 ncm-et-10 2023-06-11 19:12 01:24:52 1942255 500 427 0 73 +441.5 +/- 40.29
191594 ncm-et-9 2023-06-11 19:06 01:25:46 1949833 500 430 1 69 +446.7 +/- 41.57
191587 ncm-et-9 2023-06-11 17:40 01:25:26 1942991 500 419 0 81 +421.93 +/- 38.15
191586 ncm-et-4 2023-06-11 17:27 01:46:56 1562470 500 428 0 72 +444.08 +/- 40.59
191585 ncm-et-13 2023-06-11 17:27 01:44:48 1568965 500 422 1 77 +426.65 +/- 39.25
191584 ncm-et-15 2023-06-11 17:27 01:45:09 1606590 500 425 0 75 +436.43 +/- 39.73
191583 ncm-et-10 2023-06-11 17:27 01:44:22 1589883 500 437 0 63 +468.95 +/- 43.54
191582 ncm-et-9 2023-06-11 17:27 00:09:43 1954549 55 47 0 8 +442.16 +/- 140.62
191581 ncm-et-3 2023-06-11 17:27 01:45:51 1586932 500 438 0 62 +471.92 +/- 43.9
191580 ncm-et-13 2023-06-11 17:00 00:10:00 1962009 58 49 0 9 +430.02 +/- 129.33
191579 ncm-et-10 2023-06-11 17:00 00:09:54 1943438 54 48 0 6 +492.12 +/- 178.49
191578 ncm-et-4 2023-06-11 17:00 00:09:56 1952220 56 49 0 7 +470.39 +/- 156.32
191577 ncm-et-3 2023-06-11 17:00 00:10:08 1966318 58 44 0 14 +344.96 +/- 96.46
191576 ncm-et-15 2023-06-11 17:00 00:09:58 1952074 58 50 0 8 +452.09 +/- 141.0
191575 ncm-et-9 2023-06-11 17:00 00:10:10 1953742 55 47 0 8 +442.16 +/- 140.62

Commit

Commit ID 932f5a2d657c846c282adcf2051faef7ca17ae15
Author Linmiao Xu
Date 2023-06-11 13:23:52 UTC
Update default net to nn-ea57bea57e32.nnue Created by retraining an earlier epoch (ep659) of the experiment that led to the first SFNNv6 net: - First retrained on the nn-0dd1cebea573 dataset - Then retrained with skip 20 on a smaller dataset containing unfiltered Leela data - And then retrained again with skip 27 on the nn-0dd1cebea573 dataset The equivalent 7-step training sequence from scratch that led here was: 1. max-epoch 400, lambda 1.0, constant LR 9.75e-4, T79T77-filter-v6-dd.min.binpack ep379 chosen for retraining in step2 2. max-epoch 800, end-lambda 0.75, T60T70wIsRightFarseerT60T74T75T76.binpack ep679 chosen for retraining in step3 3. max-epoch 800, end-lambda 0.75, skip 28, nn-e1fb1ade4432 dataset ep799 chosen for retraining in step4 4. max-epoch 800, end-lambda 0.7, skip 28, nn-e1fb1ade4432 dataset ep759 became nn-8d69132723e2.nnue (first SFNNv6 net) ep659 chosen for retraining in step5 5. max-epoch 800, end-lambda 0.7, skip 28, nn-0dd1cebea573 dataset ep759 chosen for retraining in step6 6. max-epoch 800, end-lambda 0.7, skip 20, leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack ep639 chosen for retraining in step7 7. max-epoch 800, end-lambda 0.7, skip 27, nn-0dd1cebea573 dataset ep619 became nn-ea57bea57e32.nnue For the last retraining (step7): python3 easy_train.py --experiment-name L1-1536-Re6-masterShuffled-ep639-sk27-Re5-leela-dfrc-v2-T77toT80small-Re4-masterShuffled-ep659-Re3-sameAs-Re2-leela96-dfrc99-16t-v2-T60novdecT80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-Re1-LeelaFarseer-new-T77T79 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --early-fen-skipping 27 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 800 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-Re5-leela-dfrc-v2-T77toT80small-epoch639.nnue \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --seed $RANDOM \ --gpus "0," For preparing the step6 leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack dataset: python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-16tb7p-eval-filt-v2.binpack \ test77-dec2021-16tb7p.no-db.min-mar2023.binpack \ test78-janfeb2022-16tb7p.no-db.min-mar2023.binpack \ test79-apr2022-16tb7p-filter-v6-dd.binpack \ test80-apr2022-16tb7p.no-db.min-mar2023.binpack \ /data/leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack The unfiltered Leela data used for the step6 dataset can be found at: https://robotmoon.com/nnue-training-data Local elo at 25k nodes per move: nn-epoch619.nnue : 2.3 +/- 1.9 Passed STC: https://tests.stockfishchess.org/tests/view/6480d43c6e6ce8d9fc6d7cc8 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 40992 W: 11017 L: 10706 D: 19269 Ptnml(0-2): 113, 4400, 11170, 4689, 124 Passed LTC: https://tests.stockfishchess.org/tests/view/648119ac6e6ce8d9fc6d8208 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 129174 W: 35059 L: 34579 D: 59536 Ptnml(0-2): 66, 12548, 38868, 13050, 55 closes https://github.com/official-stockfish/Stockfish/pull/4611 bench: 2370027
Copyright 2011–2024 Next Chess Move LLC