NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.
| Host | Duration | Avg Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo |
|---|---|---|---|---|---|---|---|
| ncm-dbt-01 | 01:42:51 | 583935 | 1000 | 261 243 496 | +6.25 ± 10.27 | 1 100 281 116 2 | +11.82 ± 20.17 |
| ncm-dbt-02 | 01:42:56 | 587878 | 1000 | 251 237 512 | +4.86 ± 10.89 | 0 119 250 129 2 | +8.34 ± 21.56 |
| ncm-dbt-03 | 01:42:42 | 588047 | 1000 | 272 242 486 | +10.43 ± 10.7 | 0 106 261 130 3 | +18.78 ± 21.08 |
| ncm-dbt-04 | 01:43:18 | 572864 | 1000 | 271 217 512 | +18.78 ± 10.09 | 0 85 276 139 0 | +37.67 ± 20.38 |
| ncm-dbt-05 | 01:43:29 | 586395 | 1000 | 253 247 500 | +2.08 ± 10.46 | 1 111 270 117 1 | +4.17 ± 20.68 |
| 5000 | 1308 1186 2506 | +8.48 ± 4.69 | 2 521 1338 631 8 | +16.13 ± 9.29 | |||
| ID | Host | Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo | CLI | PGN | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 398148 | ncm-dbt-04 | 571391 | 500 | 130 112 258 | +12.51 ± 14.74 | 0 50 132 68 0 | +25.06 ± 29.65 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3052576166 \
-pgnout ncm-dbt-20220515-1820-010.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398147 | ncm-dbt-02 | 587113 | 500 | 122 119 259 | +2.08 ± 15.36 | 0 62 123 65 0 | +4.17 ± 30.78 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1881370322 \
-pgnout ncm-dbt-20220515-1820-009.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398146 | ncm-dbt-05 | 585084 | 500 | 131 123 246 | +5.56 ± 14.41 | 1 49 141 59 0 | +12.51 ± 28.5 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3554065129 \
-pgnout ncm-dbt-20220515-1820-008.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398145 | ncm-dbt-03 | 587367 | 500 | 140 122 238 | +12.51 ± 16.07 | 0 60 113 76 1 | +23.66 ± 31.99 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 131629909 \
-pgnout ncm-dbt-20220515-1820-007.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398144 | ncm-dbt-01 | 581985 | 500 | 128 121 251 | +4.87 ± 14.48 | 1 50 140 59 0 | +11.12 ± 28.63 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 486653825 \
-pgnout ncm-dbt-20220515-1820-006.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398143 | ncm-dbt-04 | 574337 | 500 | 141 105 254 | +25.06 ± 13.75 | 0 35 144 71 0 | +50.38 ± 27.99 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 600976176 \
-pgnout ncm-dbt-20220515-1820-005.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398142 | ncm-dbt-02 | 588643 | 500 | 129 118 253 | +7.64 ± 15.46 | 0 57 127 64 2 | +12.51 ± 30.29 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 295026215 \
-pgnout ncm-dbt-20220515-1820-004.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398141 | ncm-dbt-01 | 585885 | 500 | 133 122 245 | +7.64 ± 14.59 | 0 50 141 57 2 | +12.51 ± 28.5 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1824015123 \
-pgnout ncm-dbt-20220515-1820-003.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398140 | ncm-dbt-03 | 588728 | 500 | 132 120 248 | +8.34 ± 14.13 | 0 46 148 54 2 | +13.9 ± 27.56 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3716337443 \
-pgnout ncm-dbt-20220515-1820-002.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 398139 | ncm-dbt-05 | 587707 | 500 | 122 124 254 | -1.39 ± 15.17 | 0 62 129 58 1 | -4.17 ± 30.04 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1040812122 \
-pgnout ncm-dbt-20220515-1820-001.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20220515-1820 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:22b7909809c731aea691184dd7c1a2b02c5946af \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| Commit ID | 22b7909809c731aea691184dd7c1a2b02c5946af |
|---|---|
| Author | xoto10 |
| Date | 2022-05-15 18:20:37 UTC |
|
Tune scale and optimism.
Tune scale and optimism in effort to make stockfish play more aggressively.
STC @ 10+0.1 th 1:
LLR: 2.94 (-2.94,2.94) <0.00,2.50>
Total: 27896 W: 7506 L: 7248 D: 13142
Ptnml(0-2): 103, 3047, 7388, 3309, 101
https://tests.stockfishchess.org/tests/live_elo/627fd0cfab44257388ab1f13
LTC @ 60+0.6 th 1:
LLR: 2.93 (-2.94,2.94) <0.50,3.00>
Total: 65576 W: 17512 L: 17178 D: 30886
Ptnml(0-2): 37, 6397, 19587, 6729, 38
https://tests.stockfishchess.org/tests/live_elo/627ff666ab44257388ab256d
closes https://github.com/official-stockfish/Stockfish/pull/4025
Bench 6407734
|
|