Dev Builds » 20200811-0617

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo
ncm-dbt-01	06:33:26	585156	3992	52 2586 1354	-260.36 ± 6.63	611 1312 73 0 0	-691.94 ± 40.53
ncm-dbt-02	06:32:06	585470	4000	78 2613 1309	-259.76 ± 6.98	639 1258 102 1 0	-629.43 ± 34.11
ncm-dbt-03	06:32:22	584543	4000	58 2611 1331	-262.39 ± 6.81	632 1291 75 2 0	-678.31 ± 39.95
ncm-dbt-04	06:32:31	567743	4020	50 2621 1349	-263.15 ± 6.8	639 1294 76 1 0	-681.45 ± 39.69
ncm-dbt-05	06:33:04	581493	3988	67 2633 1288	-265.44 ± 7.08	663 1240 91 0 0	-652.67 ± 36.16
			20000	305 13064 6631	-262.21 ± 3.07	3184 6395 417 4 0	-665.33 ± 16.72

Test Detail

ID	Host	Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo	PGN
461531	ncm-dbt-04	568395	20	1 9 10	-147.12 ± 52.02	0 8 2 0 0	-381.52 ± 492.07	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3740282176 \ -pgnout ncm-dbt-20200811-0617-041.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461530	ncm-dbt-05	581652	488	9 329 150	-272.83 ± 20.88	86 148 10 0 0	-671.74 ± 124.52	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3097250537 \ -pgnout ncm-dbt-20200811-0617-040.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461529	ncm-dbt-01	584453	492	7 318 167	-258.81 ± 19.03	75 161 10 0 0	-673.18 ± 124.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 626611355 \ -pgnout ncm-dbt-20200811-0617-039.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461528	ncm-dbt-02	584537	500	12 338 150	-270.57 ± 21.1	89 148 13 0 0	-629.41 ± 104.8	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4118795463 \ -pgnout ncm-dbt-20200811-0617-038.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461527	ncm-dbt-03	584958	500	9 321 170	-254.16 ± 19.31	75 163 11 1 0	-629.43 ± 115.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4126742490 \ -pgnout ncm-dbt-20200811-0617-037.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461526	ncm-dbt-04	565312	500	8 320 172	-254.16 ± 19.82	77 159 13 1 0	-603.86 ± 103.96	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2763041874 \ -pgnout ncm-dbt-20200811-0617-036.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461525	ncm-dbt-05	584453	500	8 321 171	-255.29 ± 18.66	74 165 11 0 0	-659.13 ± 116.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1077393673 \ -pgnout ncm-dbt-20200811-0617-035.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461524	ncm-dbt-02	585717	500	7 316 177	-250.76 ± 18.08	70 169 11 0 0	-659.13 ± 116.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 538618998 \ -pgnout ncm-dbt-20200811-0617-034.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461523	ncm-dbt-01	584579	500	6 326 168	-263.41 ± 18.55	77 166 7 0 0	-739.05 ± 163.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2489312062 \ -pgnout ncm-dbt-20200811-0617-033.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461522	ncm-dbt-03	584579	500	6 329 165	-266.96 ± 19.28	81 161 8 0 0	-715.51 ± 146.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1561561303 \ -pgnout ncm-dbt-20200811-0617-032.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461521	ncm-dbt-04	569390	500	3 336 161	-279.16 ± 19.54	87 159 4 0 0	-837.28 ± 240.94	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3769969701 \ -pgnout ncm-dbt-20200811-0617-031.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461520	ncm-dbt-05	582611	500	7 336 157	-274.22 ± 20.15	87 155 8 0 0	-715.51 ± 146.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2885090756 \ -pgnout ncm-dbt-20200811-0617-030.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461519	ncm-dbt-02	584790	500	16 338 146	-265.78 ± 22.05	90 143 16 1 0	-571.11 ± 91.87	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1086272685 \ -pgnout ncm-dbt-20200811-0617-029.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461518	ncm-dbt-01	584579	500	5 318 177	-255.29 ± 17.25	69 175 6 0 0	-766.17 ± 188.02	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2925902861 \ -pgnout ncm-dbt-20200811-0617-028.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461517	ncm-dbt-03	585801	500	5 332 163	-271.78 ± 19.57	84 159 7 0 0	-739.05 ± 163.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 383328903 \ -pgnout ncm-dbt-20200811-0617-027.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461516	ncm-dbt-04	570029	500	8 328 164	-263.41 ± 19.12	79 162 9 0 0	-694.7 ± 134.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2385293186 \ -pgnout ncm-dbt-20200811-0617-026.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461515	ncm-dbt-02	587240	500	9 333 158	-268.16 ± 21.07	88 148 14 0 0	-616.18 ± 100.06	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2908730854 \ -pgnout ncm-dbt-20200811-0617-025.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461514	ncm-dbt-01	583656	500	8 334 158	-270.57 ± 20.56	87 152 11 0 0	-659.13 ± 116.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1787176528 \ -pgnout ncm-dbt-20200811-0617-024.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461513	ncm-dbt-05	581943	500	9 321 170	-254.15 ± 19.57	77 158 15 0 0	-603.84 ± 95.91	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1403737796 \ -pgnout ncm-dbt-20200811-0617-023.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461512	ncm-dbt-03	583321	500	10 319 171	-250.76 ± 18.88	73 163 14 0 0	-616.18 ± 100.06	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3488135904 \ -pgnout ncm-dbt-20200811-0617-022.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461511	ncm-dbt-04	569869	500	5 329 166	-268.16 ± 19.71	83 158 9 0 0	-694.7 ± 134.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 858648128 \ -pgnout ncm-dbt-20200811-0617-021.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461510	ncm-dbt-02	585632	500	9 328 163	-262.24 ± 20.33	83 153 14 0 0	-616.18 ± 100.06	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1082277118 \ -pgnout ncm-dbt-20200811-0617-020.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461509	ncm-dbt-01	586266	500	6 318 176	-254.15 ± 18.24	72 168 10 0 0	-676.04 ± 124.55	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3256596595 \ -pgnout ncm-dbt-20200811-0617-019.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461508	ncm-dbt-05	583908	500	6 327 167	-264.59 ± 19.27	80 161 9 0 0	-694.7 ± 134.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 583464295 \ -pgnout ncm-dbt-20200811-0617-018.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461507	ncm-dbt-03	583531	500	5 314 181	-250.76 ± 17.25	67 175 8 0 0	-715.51 ± 146.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3827269678 \ -pgnout ncm-dbt-20200811-0617-017.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461506	ncm-dbt-04	567363	500	5 322 173	-259.9 ± 19.24	78 161 11 0 0	-659.13 ± 116.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 332399431 \ -pgnout ncm-dbt-20200811-0617-016.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461505	ncm-dbt-02	584201	500	9 313 178	-245.19 ± 18.41	69 166 15 0 0	-603.84 ± 95.91	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3610704903 \ -pgnout ncm-dbt-20200811-0617-015.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461504	ncm-dbt-05	580572	500	12 322 166	-251.89 ± 19.02	74 162 14 0 0	-616.18 ± 100.06	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2084221120 \ -pgnout ncm-dbt-20200811-0617-014.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461503	ncm-dbt-01	586985	500	6 330 164	-268.16 ± 19.99	84 156 10 0 0	-676.04 ± 124.55	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 681198006 \ -pgnout ncm-dbt-20200811-0617-013.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461502	ncm-dbt-03	583698	500	6 353 141	-297.27 ± 22.89	105 137 8 0 0	-715.51 ± 146.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 252603420 \ -pgnout ncm-dbt-20200811-0617-012.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461501	ncm-dbt-04	565233	500	6 323 171	-259.9 ± 18.69	76 165 9 0 0	-694.7 ± 134.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2823936731 \ -pgnout ncm-dbt-20200811-0617-011.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461500	ncm-dbt-02	585928	500	9 324 167	-257.59 ± 18.95	76 163 11 0 0	-659.13 ± 116.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3117589807 \ -pgnout ncm-dbt-20200811-0617-010.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461499	ncm-dbt-05	578465	500	9 330 161	-264.59 ± 20.09	83 155 12 0 0	-643.67 ± 110.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 367107835 \ -pgnout ncm-dbt-20200811-0617-009.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461498	ncm-dbt-03	585421	500	10 317 173	-248.52 ± 18.85	71 166 12 1 0	-616.2 ± 109.19	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 625941526 \ -pgnout ncm-dbt-20200811-0617-008.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461497	ncm-dbt-01	587367	500	8 318 174	-251.89 ± 17.95	70 170 10 0 0	-676.04 ± 124.55	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4040542718 \ -pgnout ncm-dbt-20200811-0617-007.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461496	ncm-dbt-04	565744	500	3 327 170	-268.16 ± 19.13	81 162 7 0 0	-739.05 ± 163.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3529565644 \ -pgnout ncm-dbt-20200811-0617-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461495	ncm-dbt-02	585717	500	7 323 170	-258.74 ± 18.26	74 168 8 0 0	-715.51 ± 146.53	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1723820195 \ -pgnout ncm-dbt-20200811-0617-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461494	ncm-dbt-03	585042	500	7 326 167	-262.24 ± 18.41	76 167 7 0 0	-739.05 ± 163.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2348862121 \ -pgnout ncm-dbt-20200811-0617-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461493	ncm-dbt-05	578342	500	7 347 146	-288.06 ± 22.97	102 136 12 0 0	-643.67 ± 110.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3368670497 \ -pgnout ncm-dbt-20200811-0617-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461492	ncm-dbt-04	568355	500	11 327 162	-258.74 ± 19.36	78 160 12 0 0	-643.67 ± 110.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2097750035 \ -pgnout ncm-dbt-20200811-0617-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
461491	ncm-dbt-01	583363	500	6 324 170	-261.07 ± 18.83	77 164 9 0 0	-694.7 ± 134.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3119720330 \ -pgnout ncm-dbt-20200811-0617-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20200811-0617 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:ea6220f3813e5b76b444a02905eaf2c556bdb368 \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15

Commit

Commit ID	ea6220f3813e5b76b444a02905eaf2c556bdb368
Author	Guy Vreuls
Date	2020-08-11 06:17:03 UTC
This commit enables a mixed bench, to improve CI and allow for PGO (profile-build) of the NNUE part of the code. Joint work gvreuls / vondele * Download the default NNUE net in AppVeyor * Download net in travis CI `make net` * Adjust tests to cover more archs, speedup instrumented testing * Introduce 'mixed' bench as default, with further options: classical, NNUE, mixed. mixed (default) and NNUE require the default net to be present, which can be obtained with ``` make net ``` Further examples (first is equivalent to `./stockfish bench`): ``` ./stockfish bench 16 1 13 default depth mixed ./stockfish bench 16 1 13 default depth classical ./stockfish bench 16 1 13 default depth NNUE ``` The net is now downloaded automatically if needed for `profile-build` (usual `build` works fine without net present) PGO gives a nice speedup on fishtest: passed STC: LLR: 2.93 (-2.94,2.94) {-0.50,1.50} Total: 3360 W: 469 L: 343 D: 2548 Ptnml(0-2): 20, 246, 1030, 356, 28 https://tests.stockfishchess.org/tests/view/5f31b5499081672066537569 passed LTC: LLR: 2.97 (-2.94,2.94) {0.25,1.75} Total: 8824 W: 609 L: 502 D: 7713 Ptnml(0-2): 8, 430, 3438, 519, 17 https://tests.stockfishchess.org/tests/view/5f31c87b908167206653757c closes https://github.com/official-stockfish/Stockfish/pull/2931 fixes https://github.com/official-stockfish/Stockfish/issues/2907 requires fishtest updates before commit Bench: 4290577