Dev Builds » 20220917-0713

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 13:30:34 1962109 4041 3459 11 571 +440.55 ± 14.26
ncm-et-9 13:30:51 1963629 4033 3447 9 577 +439.54 ± 14.18
ncm-et-10 13:11:40 1954325 3876 3289 8 579 +432.09 ± 14.15
ncm-et-13 13:31:16 1962808 4049 3472 5 572 +444.43 ± 14.24
ncm-et-15 13:30:42 1957600 4001 3422 7 572 +440.91 ± 14.24
20000 17089 40 2871 +439.52 ± 6.35

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
160683 ncm-et-15 2022-09-18 14:53 00:00:58 1959244 1 0 0 1 0.0 ± 30.47
160682 ncm-et-9 2022-09-18 14:47 00:07:39 1964313 33 30 0 3 +528.77 ± 403.68
160681 ncm-et-3 2022-09-18 14:46 00:08:51 1963852 41 38 0 3 +568.09 ± 383.55
160680 ncm-et-13 2022-09-18 14:44 00:10:29 1961534 49 42 0 7 +445.53 ± 155.11
160679 ncm-et-10 2022-09-18 14:43 00:11:07 1954499 49 46 0 3 +600.13 ± 367.21
160678 ncm-et-10 2022-09-18 14:09 00:31:04 1955422 150 140 0 10 +584.92 ± 123.67
160677 ncm-et-15 2022-09-18 13:15 01:38:08 1964920 500 434 1 65 +457.52 ± 42.89
160676 ncm-et-9 2022-09-18 13:07 01:39:23 1962148 500 429 0 71 +446.7 ± 40.89
160675 ncm-et-3 2022-09-18 13:03 01:42:16 1953615 500 429 2 69 +441.5 ± 41.61
160674 ncm-et-13 2022-09-18 13:03 01:41:06 1961087 500 441 1 58 +477.99 ± 45.54
160673 ncm-et-10 2022-09-18 12:49 01:16:43 1927830 374 319 1 54 +436.77 ± 47.19
160672 ncm-et-15 2022-09-18 11:34 01:40:14 1955574 500 432 0 68 +454.76 ± 41.82
160671 ncm-et-9 2022-09-18 11:26 01:39:57 1966184 500 421 3 76 +419.61 ± 39.58
160670 ncm-et-3 2022-09-18 11:24 01:38:48 1959392 500 422 0 78 +429.05 ± 38.92
160669 ncm-et-13 2022-09-18 11:21 01:40:50 1961076 500 424 2 74 +429.05 ± 40.11
160668 ncm-et-10 2022-09-18 11:05 01:43:13 1947727 500 426 2 72 +433.94 ± 40.69
160667 ncm-et-10 2022-09-18 10:36 00:25:31 1957863 122 108 0 14 +486.22 ± 98.99
160666 ncm-et-10 2022-09-18 10:20 00:12:27 1963072 58 48 0 10 +410.09 ± 120.16
160665 ncm-et-15 2022-09-18 09:52 01:40:57 1963543 500 433 0 67 +457.52 ± 42.15
160664 ncm-et-9 2022-09-18 09:44 01:41:13 1960461 500 424 3 73 +426.65 ± 40.41
160663 ncm-et-3 2022-09-18 09:43 01:39:58 1959285 500 428 2 70 +438.95 ± 41.29
160662 ncm-et-13 2022-09-18 09:42 01:38:38 1968624 500 428 0 72 +444.08 ± 40.59
160661 ncm-et-10 2022-09-18 08:38 01:41:52 1952070 500 416 2 82 +410.58 ± 38.02
160660 ncm-et-15 2022-09-18 08:12 01:39:56 1960618 500 425 3 72 +429.05 ± 40.7
160659 ncm-et-3 2022-09-18 08:03 01:38:52 1970024 500 427 2 71 +436.43 ± 40.99
160658 ncm-et-9 2022-09-18 08:03 01:40:56 1960614 500 427 0 73 +441.5 ± 40.29
160657 ncm-et-13 2022-09-18 08:01 01:40:09 1962008 500 428 0 72 +444.08 ± 40.59
160656 ncm-et-10 2022-09-18 06:59 01:38:27 1963706 500 420 1 79 +421.93 ± 38.73
160655 ncm-et-15 2022-09-18 06:28 01:43:02 1957859 500 422 1 77 +426.65 ± 39.25
160654 ncm-et-9 2022-09-18 06:23 01:39:16 1963687 500 434 0 66 +460.32 ± 42.48
160653 ncm-et-3 2022-09-18 06:22 01:40:28 1959406 500 441 1 58 +477.99 ± 45.54
160652 ncm-et-13 2022-09-18 06:19 01:41:18 1958013 500 427 0 73 +441.5 ± 40.29
160651 ncm-et-10 2022-09-18 05:15 01:42:46 1959086 500 423 0 77 +431.48 ± 39.18
160650 ncm-et-15 2022-09-18 04:45 01:42:22 1944415 500 427 0 73 +441.5 ± 40.29
160649 ncm-et-9 2022-09-18 04:42 01:40:14 1964301 500 421 1 78 +424.28 ± 38.99
160648 ncm-et-3 2022-09-18 04:41 01:40:42 1969566 500 423 1 76 +429.05 ± 39.52
160647 ncm-et-13 2022-09-18 04:39 01:39:18 1963849 500 426 1 73 +436.43 ± 40.36
160646 ncm-et-10 2022-09-18 03:34 01:40:15 1965842 500 423 1 76 +429.05 ± 39.52
160645 ncm-et-15 2022-09-18 03:03 01:41:15 1954975 500 421 0 79 +426.65 ± 38.66
160644 ncm-et-3 2022-09-18 03:00 01:40:06 1964306 500 438 2 60 +466.04 ± 44.75
160643 ncm-et-9 2022-09-18 03:00 01:41:11 1968004 500 431 1 68 +449.35 ± 41.89
160642 ncm-et-13 2022-09-18 02:58 01:40:28 1960926 500 420 1 79 +421.93 ± 38.73
160641 ncm-et-10 2022-09-18 01:51 01:42:20 1948244 500 423 0 77 +431.48 ± 39.18
160640 ncm-et-10 2022-09-18 01:34 00:13:46 1951317 65 51 0 14 +367.31 ± 97.01
160639 ncm-et-3 2022-09-18 01:19 01:40:33 1959542 500 413 1 86 +406.2 ± 37.04
160638 ncm-et-10 2022-09-18 01:19 00:12:09 1959553 58 46 1 11 +359.56 ± 112.62
160637 ncm-et-15 2022-09-18 01:18 01:43:50 1957253 500 428 2 70 +438.95 ± 41.29
160636 ncm-et-13 2022-09-18 01:18 01:39:00 1968159 500 436 0 64 +466.03 ± 43.18
160635 ncm-et-9 2022-09-18 01:18 01:41:02 1962951 500 430 1 69 +446.7 ± 41.57

Commit

Commit ID 154e7afed0fe9c6f45a2aee8ef6f38d44076cb19
Author atumanian
Date 2022-09-17 07:13:07 UTC
Simplify trend and optimism. This patch simplifies the formulas used to compute the trend and optimism values before each search iteration. As a side effect, this removes the parameters which make the relationship between the displayed evaluation value and the expected game result asymmetric. I've also provided links to the results of isotonic regression analysis of the relationship between the evaluation and game result (statistical data and a graph) for both tests, which demonstrate that the new version has a more symmetric relationship: STC: [Data and graph](https://github.com/official-stockfish/Stockfish/discussions/4150#discussioncomment-3548954) LTC: [Data and graph](https://github.com/official-stockfish/Stockfish/discussions/4150#discussioncomment-3626311) See also https://github.com/official-stockfish/Stockfish/issues/4142 passed STC: https://tests.stockfishchess.org/tests/view/6313f44b8202a039920e27e6 LLR: 2.96 (-2.94,2.94) <-1.75,0.25> Total: 108016 W: 28903 L: 28760 D: 50353 Ptnml(0-2): 461, 12075, 28850, 12104, 518 passed LTC: https://tests.stockfishchess.org/tests/view/631de45db85daa436625dfe6 LLR: 3.01 (-2.94,2.94) <-1.75,0.25> Total: 34792 W: 9412 L: 9209 D: 16171 Ptnml(0-2): 24, 3374, 10397, 3577, 24 Furthermore, this does not measurably impact Elo strength against weaker engines, as demonstrated in a match of master and patch vs SF13: This patch vs SF 13: https://tests.stockfishchess.org/tests/view/631fa34ae1612778c344c6eb Elo: 141.66 +-1.2 (95%) LOS: 100.0% Total: 100000 W: 48182 L: 9528 D: 42290 Ptnml(0-2): 96, 1426, 13277, 30130, 5071 nElo: 284.13 +-3.3 (95%) PairsRatio: 23.13 Master vs SF 13: https://tests.stockfishchess.org/tests/view/631fa3ece1612778c344c6ff Elo: 143.26 +-1.2 (95%) LOS: 100.0% Total: 100000 W: 48525 L: 9479 D: 41996 Ptnml(0-2): 94, 1537, 13098, 29771, 5500 nElo: 281.70 +-3.3 (95%) PairsRatio: 21.63 closes: https://github.com/official-stockfish/Stockfish/pull/4163 Bench: 4425574
Copyright 2011–2024 Next Chess Move LLC