Dev Builds » 20150403-0240

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 05:54:22 1985185 2410 190 505 1715 -45.67 +/- 7.35
ncm-et-4 05:54:03 1976047 2402 201 503 1698 -43.91 +/- 7.44
ncm-et-5 01:47:42 2003467 736 65 169 502 -49.42 +/- 13.99
ncm-et-6 01:47:31 2026311 728 73 156 499 -39.78 +/- 14.05
ncm-et-7 01:47:39 1978328 730 71 174 485 -49.35 +/- 14.46
ncm-et-8 01:47:29 1957656 705 54 137 514 -41.09 +/- 13.19
ncm-et-9 05:54:15 1987356 2437 193 526 1718 -47.77 +/- 7.39
ncm-et-10 05:54:05 1986136 2401 174 504 1723 -48.06 +/- 7.27
ncm-et-11 01:47:17 2013382 727 53 168 506 -55.42 +/- 13.69
ncm-et-12 01:47:37 2019941 731 70 154 507 -40.1 +/- 13.83
ncm-et-13 05:17:10 1985221 2127 161 469 1497 -50.67 +/- 7.91
ncm-et-14 01:47:20 2018070 739 45 160 534 -54.51 +/- 12.92
ncm-et-15 05:54:25 1990701 2391 187 524 1680 -49.3 +/- 7.49
ncm-et-16 01:47:42 2022467 736 53 148 535 -45.1 +/- 12.93
20000 1590 4297 14113 -47.32 +/- 2.58

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
75572 ncm-et-13 2019-03-28 10:58 00:21:00 1975612 143 18 27 98 -21.9 +/- 31.96 ↓
75571 ncm-et-15 2019-03-28 10:56 00:23:35 1981668 159 13 30 116 -37.29 +/- 27.86 ↓
75570 ncm-et-3 2019-03-28 10:55 00:24:42 1967385 166 10 39 117 -61.33 +/- 28.11 ↓
75569 ncm-et-4 2019-03-28 10:55 00:24:39 1947263 165 10 29 126 -40.19 +/- 25.42 ↓
75568 ncm-et-10 2019-03-28 10:55 00:24:57 1980627 168 12 29 127 -35.28 +/- 25.71 ↓
75567 ncm-et-9 2019-03-28 10:50 00:29:17 1966465 199 12 41 146 -50.99 +/- 24.46 ↓
75566 ncm-et-13 2019-03-28 09:42 01:15:22 1978726 500 27 119 354 -64.66 +/- 16.03 ↓
75565 ncm-et-15 2019-03-28 09:42 01:13:16 1987562 500 48 106 346 -40.48 +/- 16.77 ↓
75564 ncm-et-3 2019-03-28 09:40 01:14:02 1966310 500 39 95 366 -39.08 +/- 15.6 ↓
75563 ncm-et-4 2019-03-28 09:39 01:14:50 1961995 500 45 106 349 -42.6 +/- 16.58 ↓
75562 ncm-et-10 2019-03-28 09:39 01:14:54 1963231 500 40 115 345 -52.51 +/- 16.72 ↓
75561 ncm-et-9 2019-03-28 09:37 01:12:18 1962323 500 46 126 328 -56.07 +/- 17.65 ↓
75560 ncm-et-13 2019-03-28 08:26 01:15:15 1967695 500 46 115 339 -48.25 +/- 17.1 ↓
75559 ncm-et-4 2019-03-28 08:26 01:12:15 1966460 500 28 94 378 -46.13 +/- 14.76 ↓
75558 ncm-et-15 2019-03-28 08:25 01:15:31 1970964 500 33 115 352 -57.5 +/- 16.25 ↓
75557 ncm-et-9 2019-03-28 08:24 01:11:47 1973594 500 37 93 370 -39.08 +/- 15.35 ↓
75556 ncm-et-3 2019-03-28 08:24 01:14:46 1962467 500 34 97 369 -44.01 +/- 15.36 ↓
75555 ncm-et-10 2019-03-28 08:23 01:14:33 1965082 500 32 110 358 -54.65 +/- 15.92 ↓
75554 ncm-et-10 2019-03-28 07:10 01:12:14 1968158 500 26 94 380 -47.55 +/- 14.61 ↓
75553 ncm-et-13 2019-03-28 07:10 01:14:57 1979972 500 34 106 360 -50.38 +/- 15.85 ↓
75552 ncm-et-4 2019-03-28 07:10 01:14:53 1958481 500 41 108 351 -46.84 +/- 16.42 ↓
75551 ncm-et-9 2019-03-28 07:10 01:13:34 1974212 500 42 101 357 -41.19 +/- 16.12 ↓
75550 ncm-et-3 2019-03-28 07:09 01:13:30 1964457 500 45 108 347 -44.01 +/- 16.68 ↓
75549 ncm-et-15 2019-03-28 07:09 01:14:31 1967389 500 41 108 351 -46.84 +/- 16.42 ↓
14850 ncm-et-8 2018-04-03 08:50 00:30:27 2003725 205 15 43 147 -47.75 +/- 24.96 ↓
14849 ncm-et-11 2018-04-03 08:47 00:33:05 2013054 227 23 54 150 -47.75 +/- 26.13 ↓
14848 ncm-et-12 2018-04-03 08:47 00:33:20 2019045 231 25 48 158 -34.71 +/- 25.08 ↓
14847 ncm-et-7 2018-04-03 08:47 00:33:43 1976804 230 17 57 156 -61.04 +/- 25.05 ↓
14846 ncm-et-4 2018-04-03 08:47 00:33:46 2024101 237 24 47 166 -33.82 +/- 24.09 ↓
14845 ncm-et-6 2018-04-03 08:47 00:33:49 2028196 228 26 49 153 -35.17 +/- 25.77 ↓
14844 ncm-et-15 2018-04-03 08:47 00:34:03 2018883 232 17 51 164 -51.29 +/- 23.85 ↓
14843 ncm-et-5 2018-04-03 08:46 00:34:21 2023447 236 22 46 168 -35.45 +/- 23.64 ↓
14842 ncm-et-14 2018-04-03 08:46 00:34:13 2017258 239 20 52 167 -46.8 +/- 23.91 ↓
14841 ncm-et-10 2018-04-03 08:46 00:34:42 2020348 233 18 43 172 -37.42 +/- 22.61 ↓
14840 ncm-et-16 2018-04-03 08:45 00:35:15 2022140 236 12 47 177 -51.91 +/- 21.67 ↓
14839 ncm-et-3 2018-04-03 08:45 00:35:00 2023609 244 21 56 167 -50.18 +/- 24.22 ↓
14838 ncm-et-9 2018-04-03 08:45 00:35:26 2023773 238 20 53 165 -48.49 +/- 24.17 ↓
14837 ncm-et-13 2018-04-03 08:10 01:10:36 2024100 484 36 102 346 -47.67 +/- 16.3 ↓
14836 ncm-et-11 2018-04-03 07:32 01:14:12 2013711 500 30 114 356 -58.93 +/- 15.98 ↓
14835 ncm-et-10 2018-04-03 07:32 01:12:45 2019372 500 46 113 341 -46.84 +/- 17.0 ↓
14834 ncm-et-8 2018-04-03 07:32 01:17:02 1911588 500 39 94 367 -38.37 +/- 15.54 ↓
14833 ncm-et-4 2018-04-03 07:32 01:13:40 1997982 500 53 119 328 -46.13 +/- 17.73 ↓
14832 ncm-et-15 2018-04-03 07:32 01:13:29 2017745 500 35 114 351 -55.36 +/- 16.33 ↓
14831 ncm-et-3 2018-04-03 07:32 01:12:22 2026885 500 41 110 349 -48.25 +/- 16.53 ↓
14830 ncm-et-14 2018-04-03 07:32 01:13:07 2018883 500 25 108 367 -58.21 +/- 15.3 ↓
14829 ncm-et-9 2018-04-03 07:32 01:11:53 2023772 500 36 112 352 -53.22 +/- 16.3 ↓
14828 ncm-et-12 2018-04-03 07:32 01:14:17 2020837 500 45 106 349 -42.6 +/- 16.58 ↓
14827 ncm-et-5 2018-04-03 07:32 01:13:21 1983487 500 43 123 334 -56.07 +/- 17.32 ↓
14826 ncm-et-7 2018-04-03 07:32 01:13:56 1979852 500 54 117 329 -44.01 +/- 17.69 ↓
14825 ncm-et-6 2018-04-03 07:32 01:13:42 2024427 500 47 107 346 -41.89 +/- 16.76 ↓
14824 ncm-et-16 2018-04-03 07:32 01:12:27 2022795 500 41 101 358 -41.89 +/- 16.06 ↓

Commit

Commit ID 926f215061311392bc26c7bc4bde5b719dbab4e5
Author Marco Costalba
Date 2015-04-03 02:40:55 UTC
Add support for playing in 'nodes as time' mode When running more games in parallel, or simply when running a game with a background process, due to how OS scheduling works, there is no guarantee that the CPU resources allocated evenly between the two players. This introduces noise in the result that leads to unreliable result and in the worst cases can even invalidate the result. For instance in SF test framework we avoid running from clouds virtual machines because are a known source of very unstable CPU speed. To overcome this issue, without requiring changes to the GUI, the idea is to use searched nodes instead of time, and to convert time to available nodes upfront, at the beginning of the game. When nodestime UCI option is set at a given nodes per milliseconds (npmsec), at the beginning of the game (and only once), the engine reads the available time to think, sent by the GUI with 'go wtime x' UCI command. Then it translates time in available nodes (nodes = npmsec * x), then feeds available nodes instead of time to the time management logic and starts the search. During the search the engine checks the searched nodes against the available ones in such a way that all the time management logic still fully applies, and the game mimics a real one played on real time. When the search finishes, before returning best move, the total available nodes are updated, subtracting the real searched nodes. After the first move, the time information sent by the GUI is ignored, and the engine fully relies on the updated total available nodes to feed time management. To avoid time losses, the speed of the engine (npms) must be set to a value lower than real speed so that if the real TC is for instance 30 secs, and npms is half of the real speed, the game will last on average 15 secs, so much less than the TC limit, providing for a safety 'time buffer'. There are 2 main limitations with this mode. 1. Engine speed should be the same for both players, and this limits the approach to mainly parameter tuning patches. 2. Because npms is fixed while, in real engines, the speed increases toward endgame, this introduces an artifact that is equivalent to an altered time management. Namely it is like the time management gives less available time than what should be in standard case. May be the second limitation could be mitigated in a future with a smarter 'dynamic npms' approach. Tests shows that the standard deviation of the results with 'nodestime' is lower than in standard TC, as is expected because now all the introduced noise due the random speed variability of the engines during the game is fully removed. Original NIT idea by Michael Hoffman that shows how to play in NIT mode without requiring changes to the GUI. This implementation goes a bit further, the key difference is that we read TC from GUI only once upfront instead of re-reading after every move as in Michael's implementation. No functional change.
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy Desktop Version