Dev Builds » 20240917-1824

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo
ncm-dbt-01	06:52:41	585351	4000	1560 527 1913	+91.8 ± 4.88	1 91 796 1098 14	+195.51 ± 12.04
ncm-dbt-02	06:50:52	588199	4000	1525 541 1934	+87.26 ± 4.81	1 93 833 1067 6	+186.02 ± 11.75
ncm-dbt-03	06:52:29	587720	4014	1533 533 1948	+88.42 ± 4.97	1 105 813 1069 19	+185.92 ± 11.92
ncm-dbt-04	06:52:05	571809	4000	1488 577 1935	+80.54 ± 4.9	1 106 890 987 16	+167.53 ± 11.35
ncm-dbt-05	06:53:14	586060	3986	1501 548 1937	+84.71 ± 4.99	1 118 810 1055 9	+179.08 ± 11.96
			20000	7607 2726 9667	+86.54 ± 2.2	5 513 4142 5276 64	+182.7 ± 5.27

Test Detail

ID	Host	Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo	PGN
377763	ncm-dbt-03	586901	14	5 1 8	+102.04 ± 70.73	0 0 3 4 0	+225.59 ± 263.0	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1616990013 \ -pgnout ncm-dbt-20240917-1824-041.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377762	ncm-dbt-05	582861	486	186 67 233	+86.83 ± 14.05	0 13 99 130 1	+184.24 ± 34.33	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1010087132 \ -pgnout ncm-dbt-20240917-1824-040.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377761	ncm-dbt-01	584076	500	195 76 229	+84.3 ± 13.88	0 13 107 128 2	+176.33 ± 32.93	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1049874712 \ -pgnout ncm-dbt-20240917-1824-039.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377760	ncm-dbt-04	575557	500	178 71 251	+75.52 ± 14.65	0 20 105 123 2	+155.54 ± 33.38	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 740224202 \ -pgnout ncm-dbt-20240917-1824-038.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377759	ncm-dbt-02	585464	500	187 74 239	+79.9 ± 12.68	0 8 121 121 0	+169.27 ± 30.34	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3025637123 \ -pgnout ncm-dbt-20240917-1824-037.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377758	ncm-dbt-03	586985	500	195 63 242	+93.95 ± 14.23	0 12 98 136 4	+196.45 ± 34.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1030173501 \ -pgnout ncm-dbt-20240917-1824-036.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377757	ncm-dbt-05	581818	500	192 71 237	+85.78 ± 14.07	0 15 100 134 1	+181.7 ± 34.21	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2695132879 \ -pgnout ncm-dbt-20240917-1824-035.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377756	ncm-dbt-01	584790	500	194 68 238	+89.48 ± 13.22	0 9 107 133 1	+190.85 ± 32.75	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3598108399 \ -pgnout ncm-dbt-20240917-1824-034.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377755	ncm-dbt-02	587452	500	200 75 225	+88.74 ± 13.04	0 8 110 131 1	+189.0 ± 32.17	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4128618550 \ -pgnout ncm-dbt-20240917-1824-033.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377754	ncm-dbt-04	574581	500	188 79 233	+76.97 ± 13.68	0 13 117 118 2	+158.93 ± 31.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 25820561 \ -pgnout ncm-dbt-20240917-1824-032.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377753	ncm-dbt-03	585674	500	178 65 257	+79.9 ± 14.07	0 16 106 127 1	+167.53 ± 33.17	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2960572700 \ -pgnout ncm-dbt-20240917-1824-031.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377752	ncm-dbt-05	586308	500	196 82 222	+80.63 ± 15.75	1 22 93 130 4	+165.8 ± 35.41	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3114651372 \ -pgnout ncm-dbt-20240917-1824-030.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377751	ncm-dbt-04	569869	500	194 70 236	+88.0 ± 13.97	0 11 108 127 4	+181.7 ± 32.68	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 590766284 \ -pgnout ncm-dbt-20240917-1824-029.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377750	ncm-dbt-02	590653	500	195 68 237	+90.22 ± 13.86	0 13 98 138 1	+192.71 ± 34.55	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1859810144 \ -pgnout ncm-dbt-20240917-1824-028.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377749	ncm-dbt-01	585632	500	189 55 256	+95.44 ± 13.47	1 6 103 138 2	+206.01 ± 33.36	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2761931167 \ -pgnout ncm-dbt-20240917-1824-027.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377748	ncm-dbt-03	588387	500	181 77 242	+73.34 ± 14.44	1 17 110 121 1	+153.86 ± 32.54	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 801068545 \ -pgnout ncm-dbt-20240917-1824-026.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377747	ncm-dbt-05	586943	500	186 65 249	+85.78 ± 13.92	0 15 99 136 0	+183.51 ± 34.39	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4110793806 \ -pgnout ncm-dbt-20240917-1824-025.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377746	ncm-dbt-02	589155	500	181 75 244	+74.79 ± 14.2	0 18 109 122 1	+155.54 ± 32.7	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2925441019 \ -pgnout ncm-dbt-20240917-1824-024.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377745	ncm-dbt-04	569390	500	179 68 253	+78.43 ± 13.88	0 15 110 124 1	+164.07 ± 32.48	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1288169151 \ -pgnout ncm-dbt-20240917-1824-023.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377744	ncm-dbt-01	586477	500	199 67 234	+93.95 ± 14.23	0 13 95 139 3	+198.34 ± 35.12	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4102331378 \ -pgnout ncm-dbt-20240917-1824-022.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377743	ncm-dbt-03	589240	500	204 64 232	+99.95 ± 12.69	0 5 101 143 1	+217.85 ± 33.59	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 487402630 \ -pgnout ncm-dbt-20240917-1824-021.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377742	ncm-dbt-05	586816	500	188 61 251	+90.22 ± 13.55	0 11 102 136 1	+192.71 ± 33.75	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 511914799 \ -pgnout ncm-dbt-20240917-1824-020.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377741	ncm-dbt-02	587834	500	190 70 240	+85.04 ± 14.35	0 17 97 135 1	+179.9 ± 34.76	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2901060671 \ -pgnout ncm-dbt-20240917-1824-019.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377740	ncm-dbt-04	572517	500	193 70 237	+87.26 ± 14.54	1 12 104 129 4	+181.7 ± 33.45	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3162014745 \ -pgnout ncm-dbt-20240917-1824-018.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377739	ncm-dbt-03	587749	500	190 73 237	+82.83 ± 15.28	0 21 95 130 4	+169.27 ± 35.09	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1471653642 \ -pgnout ncm-dbt-20240917-1824-017.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377738	ncm-dbt-01	584495	500	197 69 234	+90.96 ± 13.87	0 13 97 139 1	+194.57 ± 34.74	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3009102625 \ -pgnout ncm-dbt-20240917-1824-016.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377737	ncm-dbt-05	587919	500	173 61 266	+79.17 ± 13.75	0 15 108 127 0	+167.53 ± 32.81	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2063653343 \ -pgnout ncm-dbt-20240917-1824-015.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377736	ncm-dbt-04	573204	500	195 72 233	+87.26 ± 12.52	0 5 118 126 1	+185.33 ± 30.57	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4160234407 \ -pgnout ncm-dbt-20240917-1824-014.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377735	ncm-dbt-02	589240	500	190 59 251	+93.2 ± 13.28	0 10 99 141 0	+202.15 ± 34.28	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 485445086 \ -pgnout ncm-dbt-20240917-1824-013.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377734	ncm-dbt-01	583028	500	200 58 242	+101.46 ± 13.04	0 8 92 150 0	+223.94 ± 35.61	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4020617575 \ -pgnout ncm-dbt-20240917-1824-012.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377733	ncm-dbt-03	587962	500	189 55 256	+95.44 ± 13.47	0 9 100 139 2	+204.07 ± 34.04	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3922855641 \ -pgnout ncm-dbt-20240917-1824-011.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377732	ncm-dbt-05	585379	500	193 64 243	+91.71 ± 12.76	0 7 107 136 0	+198.34 ± 32.63	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 679000181 \ -pgnout ncm-dbt-20240917-1824-010.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377731	ncm-dbt-04	567403	500	184 68 248	+82.1 ± 13.68	0 13 109 127 1	+172.78 ± 32.58	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1017990384 \ -pgnout ncm-dbt-20240917-1824-009.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377730	ncm-dbt-02	587240	500	188 67 245	+85.78 ± 13.61	1 9 109 130 1	+183.51 ± 32.46	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 410222097 \ -pgnout ncm-dbt-20240917-1824-008.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377729	ncm-dbt-01	586350	500	188 70 242	+83.57 ± 14.6	0 18 98 132 2	+174.55 ± 34.58	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2346425329 \ -pgnout ncm-dbt-20240917-1824-007.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377728	ncm-dbt-03	585801	500	195 60 245	+96.19 ± 13.64	0 9 100 138 3	+204.07 ± 34.04	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2865375743 \ -pgnout ncm-dbt-20240917-1824-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377727	ncm-dbt-04	571953	500	177 79 244	+68.99 ± 13.86	0 17 119 113 1	+142.26 ± 31.1	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1374198552 \ -pgnout ncm-dbt-20240917-1824-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377726	ncm-dbt-05	590438	500	187 77 236	+77.7 ± 14.72	0 20 102 126 2	+160.64 ± 33.88	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2820548377 \ -pgnout ncm-dbt-20240917-1824-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377725	ncm-dbt-02	588558	500	194 53 253	+100.7 ± 13.53	0 10 90 149 1	+219.87 ± 36.1	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3777274673 \ -pgnout ncm-dbt-20240917-1824-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377724	ncm-dbt-01	587962	500	198 64 238	+95.44 ± 13.94	0 11 97 139 3	+202.15 ± 34.7	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1232434373 \ -pgnout ncm-dbt-20240917-1824-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
377723	ncm-dbt-03	590782	500	196 75 229	+85.78 ± 14.51	0 16 100 131 3	+178.11 ± 34.22	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3605628762 \ -pgnout ncm-dbt-20240917-1824-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20240917-1824 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a06e7004c1a01fb56f5db90295884eaf3b7cd0f6 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15

Commit

Commit ID	a06e7004c1a01fb56f5db90295884eaf3b7cd0f6
Author	Disservin
Date	2024-09-17 18:24:17 UTC
Port instrumented testing to python Since an unknown amount of time the instrumented CI has been a bit flawed, explained here https://github.com/official-stockfish/Stockfish/issues/5185. It also experiences random timeout issues where restarting the workflow fixes it or very long run times (more than other workflows) and is not very portable. The intention of this commit is to port the instrumented.sh to python which also works on other operating systems. It should also be relatively easy for beginners to add new tests to assert stockfish's output and to run it. From the source directory the following command can be run. `python3 ../tests/instrumented.py --none ./stockfish` A test runner will go over the test suites and run the test cases. All instrumented tests should have been ported over. The required python version for this is should be 3.7 (untested) + the requests package, testing.py includes some infrastructure code which setups the testing. fixes https://github.com/official-stockfish/Stockfish/issues/5185 closes https://github.com/official-stockfish/Stockfish/pull/5583 No functional change