Dev Builds » 20180808-1534

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 08:38:19 1990235 3374 1721 99 1554 +182.04 ± 8.47
ncm-et-4 08:38:36 1987543 3377 1726 101 1550 +182.24 ± 8.49
ncm-et-9 08:38:30 1989086 3368 1743 84 1541 +187.43 ± 8.49
ncm-et-10 08:38:28 1950852 3316 1692 98 1526 +182.02 ± 8.55
ncm-et-13 08:38:11 1975836 3329 1752 86 1491 +191.06 ± 8.65
ncm-et-15 08:38:04 1925488 3236 1654 94 1488 +182.64 ± 8.65
20000 10288 562 9150 +184.56 ± 3.49

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
49116 ncm-et-15 2018-08-09 10:09 00:35:41 1982165 236 115 8 113 +169.88 ± 31.49
49115 ncm-et-10 2018-08-09 09:55 00:49:14 1935403 316 154 11 151 +169.51 ± 27.23
49114 ncm-et-13 2018-08-09 09:53 00:51:28 1979864 329 170 5 154 +191.55 ± 26.65
49113 ncm-et-9 2018-08-09 09:47 00:57:20 1992722 368 198 7 163 +199.78 ± 26.15
49112 ncm-et-3 2018-08-09 09:47 00:57:37 1989401 374 189 14 171 +176.29 ± 25.73
49111 ncm-et-4 2018-08-09 09:46 00:58:14 1987981 377 184 12 181 +171.13 ± 24.8
49110 ncm-et-15 2018-08-09 08:51 01:16:25 1982954 500 252 14 234 +179.9 ± 21.81
49109 ncm-et-10 2018-08-09 08:37 01:17:35 1981068 500 246 15 239 +173.67 ± 21.55
49108 ncm-et-13 2018-08-09 08:32 01:19:31 1916753 500 249 15 236 +176.33 ± 21.72
49107 ncm-et-3 2018-08-09 08:29 01:16:43 1990665 500 254 14 232 +181.7 ± 21.92
49106 ncm-et-9 2018-08-09 08:29 01:17:15 1989875 500 258 4 238 +194.57 ± 21.21
49105 ncm-et-4 2018-08-09 08:28 01:17:02 1975749 500 243 13 244 +172.78 ± 21.21
49104 ncm-et-15 2018-08-09 07:32 01:17:33 1984519 500 268 17 215 +191.78 ± 23.02
49103 ncm-et-10 2018-08-09 07:17 01:18:49 1908326 500 258 15 227 +184.42 ± 22.24
49102 ncm-et-13 2018-08-09 07:14 01:16:52 1988612 500 278 12 210 +206.01 ± 23.21
49101 ncm-et-3 2018-08-09 07:11 01:16:55 1990664 500 260 15 225 +186.25 ± 22.36
49100 ncm-et-9 2018-08-09 07:11 01:16:39 1988771 500 260 15 225 +186.25 ± 22.36
49099 ncm-et-4 2018-08-09 07:08 01:18:47 1988612 500 248 15 237 +175.44 ± 21.67
49098 ncm-et-15 2018-08-09 06:07 01:24:00 1826280 500 248 16 236 +174.55 ± 21.75
49097 ncm-et-10 2018-08-09 05:59 01:16:52 1982949 500 279 16 205 +203.11 ± 23.63
49096 ncm-et-13 2018-08-09 05:57 01:16:04 1985305 500 259 16 225 +184.42 ± 22.39
49095 ncm-et-3 2018-08-09 05:54 01:16:27 1989874 500 254 7 239 +188.08 ± 21.28
49094 ncm-et-9 2018-08-09 05:53 01:16:38 1987666 500 261 13 226 +189.0 ± 22.25
49093 ncm-et-4 2018-08-09 05:52 01:14:58 1991139 500 264 15 221 +189.92 ± 22.6
49092 ncm-et-15 2018-08-09 04:44 01:21:52 1908361 500 252 10 238 +183.51 ± 21.45
49091 ncm-et-13 2018-08-09 04:38 01:18:04 1987825 500 250 13 237 +179.0 ± 21.61
49090 ncm-et-10 2018-08-09 04:37 01:21:00 1914372 500 245 12 243 +175.44 ± 21.24
49089 ncm-et-3 2018-08-09 04:35 01:17:17 1990348 500 250 20 230 +172.78 ± 22.18
49088 ncm-et-9 2018-08-09 04:35 01:17:47 1988454 500 235 13 252 +165.8 ± 20.78
49087 ncm-et-4 2018-08-09 04:34 01:17:13 1986567 500 258 16 226 +183.51 ± 22.33
49086 ncm-et-15 2018-08-09 03:20 01:23:21 1875701 500 247 14 239 +175.44 ± 21.52
49085 ncm-et-13 2018-08-09 03:18 01:18:31 1987193 500 282 15 203 +206.98 ± 23.74
49084 ncm-et-4 2018-08-09 03:17 01:15:46 1992089 500 270 19 211 +191.78 ± 23.31
49083 ncm-et-10 2018-08-09 03:17 01:18:26 1949486 500 246 18 236 +171.02 ± 21.8
49082 ncm-et-9 2018-08-09 03:17 01:16:38 1988454 500 261 17 222 +185.33 ± 22.59
49081 ncm-et-3 2018-08-09 03:17 01:17:39 1990190 500 259 12 229 +188.08 ± 22.04
49080 ncm-et-3 2018-08-09 02:00 01:15:41 1990506 500 255 17 228 +179.9 ± 22.23
49079 ncm-et-15 2018-08-09 02:00 01:19:12 1918438 500 272 15 213 +197.4 ± 23.1
49078 ncm-et-4 2018-08-09 02:00 01:16:36 1990664 500 259 11 230 +189.0 ± 21.95
49077 ncm-et-10 2018-08-09 02:00 01:16:32 1984363 500 264 11 225 +193.64 ± 22.25
49076 ncm-et-9 2018-08-09 02:00 01:16:13 1987666 500 270 15 215 +195.51 ± 22.97
49075 ncm-et-13 2018-08-09 02:00 01:17:41 1985306 500 264 10 226 +194.57 ± 22.15

Commit

Commit ID d96c1c32a2fa109e7cc6cd07f6029cd13977121e
Author Stefano Cardanobile
Date 2018-08-08 15:34:12 UTC
Introduce voting system for best move selection Introduce voting system for best move selction in multi-threads mode. Joint work with Stefan Geschwentner, based on ideas introduced by Michael Stembera. Moves are upvoted by every thread using the margin to the minimum score across threads and the completed depth. First thread voting for the winner move is selected as best thread. Passed STC, LTC. A further LTC test with only 4 threads failed with positive score. A LTC with 31 threads was stopped with LLR 0.77 after 25k games to avoid use of excessive resources (equivalent to 1.5M STC games). Similar ideas were proposed by Michael Stembera 2 years ago #507, #508. This implementation seems simpler and more understandable, the results slightly more promising. Further possible work: 1) Tweak of the formula using for assigning votes. 2) Use a different baseline for the score dependent part: maximum score or winning probability could make more sense. 3) Assign votes in `Thread::Search` as iterations are completed and use voting results to stop search. 4) Select best thread as the threads voting for best move with the highest completed depth or, alternatively, vote on PV moves. Link to SPRT tests [stopped LTC, 31 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b61dc090ebc5902bdb95192) LLR: 0.77 (-2.94,2.94) [0.00,5.00] Total: 25602 W: 3977 L: 3850 D: 17775 Elo: 1.70 [-0.68,4.07] (95%) [passed LTC, 8 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b5df5180ebc5902bdb9162d) LLR: 2.96 (-2.94,2.94) [0.00,5.00] Total: 44478 W: 7602 L: 7300 D: 29576 Elo: 1.92 [-0.29,3.94] (95%) [failed LTC, 4 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b5f39ef0ebc5902bdb92792) LLR: -2.94 (-2.94,2.94) [0.00,5.00] Total: 29922 W: 5286 L: 5285 D: 19351 Elo: 0.48 [-1.98,3.10] (95%) [passed STC, 4 threads 5+0.05](http://tests.stockfishchess.org/tests/view/5b5dbf0f0ebc5902bdb9131c) LLR: 2.97 (-2.94,2.94) [0.00,5.00] Total: 9108 W: 2033 L: 1858 D: 5217 Elo: 6.11 [1.26,10.89] (95%) No functional change (in simple threat mode)
Copyright 2011–2024 Next Chess Move LLC