Dev Builds » 20220704-1342

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo
ncm-dbt-01	06:55:24	584104	4000	1102 952 1946	+13.03 ± 5.14	7 357 1122 507 7	+26.11 ± 10.08
ncm-dbt-02	06:55:58	585612	4010	1098 970 1942	+11.09 ± 5.14	3 382 1111 502 7	+21.51 ± 10.15
ncm-dbt-03	06:55:01	584243	4000	1075 967 1958	+9.38 ± 5.25	5 404 1074 512 5	+18.78 ± 10.36
ncm-dbt-04	06:56:17	568746	3996	1073 943 1980	+11.31 ± 5.38	9 406 1033 546 4	+23.51 ± 10.59
ncm-dbt-05	06:56:44	581229	3994	1119 968 1907	+13.14 ± 5.19	3 377 1089 522 6	+25.8 ± 10.27
			20000	5467 4800 9733	+11.59 ± 2.34	27 1926 5429 2589 29	+23.14 ± 4.6

Test Detail

ID	Host	Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo	PGN
432421	ncm-dbt-02	584916	10	3 2 5	+34.85 ± 119.9	0 1 2 2 0	+70.44 ± 307.49	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2323619787 \ -pgnout ncm-dbt-20220704-1342-041.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432420	ncm-dbt-05	577889	494	149 116 229	+23.24 ± 14.7	0 40 136 69 2	+43.84 ± 29.06	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 753547964 \ -pgnout ncm-dbt-20220704-1342-040.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432419	ncm-dbt-04	566336	496	126 127 243	-0.7 ± 15.24	2 56 131 59 0	+1.4 ± 29.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3186697042 \ -pgnout ncm-dbt-20220704-1342-039.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432418	ncm-dbt-01	583782	500	143 117 240	+18.08 ± 14.02	0 40 145 64 1	+34.86 ± 27.92	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3921518083 \ -pgnout ncm-dbt-20220704-1342-038.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432417	ncm-dbt-03	584453	500	150 117 233	+22.96 ± 14.53	0 41 136 72 1	+44.72 ± 29.1	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1919356911 \ -pgnout ncm-dbt-20220704-1342-037.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432416	ncm-dbt-02	587028	500	134 130 236	+2.78 ± 14.55	0 55 136 59 0	+5.56 ± 29.15	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1179804077 \ -pgnout ncm-dbt-20220704-1342-036.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432415	ncm-dbt-05	581610	500	137 126 237	+7.64 ± 14.84	1 50 137 61 1	+15.3 ± 29.02	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3668444346 \ -pgnout ncm-dbt-20220704-1342-035.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432414	ncm-dbt-04	569949	500	129 119 252	+6.95 ± 15.4	0 59 122 69 0	+13.9 ± 30.9	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 448875259 \ -pgnout ncm-dbt-20220704-1342-034.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432413	ncm-dbt-01	583950	500	138 128 234	+6.95 ± 14.66	2 47 140 61 0	+16.69 ± 28.62	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1903754312 \ -pgnout ncm-dbt-20220704-1342-033.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432412	ncm-dbt-03	582694	500	131 116 253	+10.43 ± 14.95	1 49 135 64 1	+20.87 ± 29.27	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 838574245 \ -pgnout ncm-dbt-20220704-1342-032.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432411	ncm-dbt-02	584201	500	143 126 231	+11.82 ± 14.81	2 44 140 63 1	+25.06 ± 28.61	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 6904169 \ -pgnout ncm-dbt-20220704-1342-031.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432410	ncm-dbt-05	583531	500	140 133 227	+4.86 ± 14.6	0 54 135 61 0	+9.73 ± 29.28	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3240920041 \ -pgnout ncm-dbt-20220704-1342-030.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432409	ncm-dbt-04	570749	500	139 119 242	+13.9 ± 15.22	0 52 127 70 1	+26.46 ± 30.28	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2925129073 \ -pgnout ncm-dbt-20220704-1342-029.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432408	ncm-dbt-01	581860	500	142 127 231	+10.43 ± 14.7	1 48 136 65 0	+22.27 ± 29.14	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3088020105 \ -pgnout ncm-dbt-20220704-1342-028.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432407	ncm-dbt-03	584159	500	137 114 249	+15.99 ± 14.12	0 43 141 66 0	+32.05 ± 28.47	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2019142655 \ -pgnout ncm-dbt-20220704-1342-027.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432406	ncm-dbt-02	586562	500	139 122 239	+11.82 ± 14.43	0 48 137 65 0	+23.66 ± 29.01	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2914022441 \ -pgnout ncm-dbt-20220704-1342-026.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432405	ncm-dbt-04	568912	500	132 111 257	+14.6 ± 15.4	2 48 127 73 0	+32.05 ± 30.28	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 137679877 \ -pgnout ncm-dbt-20220704-1342-025.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432404	ncm-dbt-01	584622	500	129 121 250	+5.56 ± 14.66	2 47 143 57 1	+12.51 ± 28.23	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1673554620 \ -pgnout ncm-dbt-20220704-1342-024.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432403	ncm-dbt-05	578465	500	127 112 261	+10.43 ± 13.78	0 44 147 59 0	+20.87 ± 27.68	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2316831559 \ -pgnout ncm-dbt-20220704-1342-023.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432402	ncm-dbt-03	585253	500	130 128 242	+1.39 ± 14.68	0 56 137 56 1	+1.39 ± 29.02	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 515406470 \ -pgnout ncm-dbt-20220704-1342-022.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432401	ncm-dbt-02	583531	500	129 123 248	+4.17 ± 14.41	1 50 141 58 0	+9.73 ± 28.5	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2994353055 \ -pgnout ncm-dbt-20220704-1342-021.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432400	ncm-dbt-04	568832	500	127 105 268	+15.3 ± 15.57	0 53 124 71 2	+27.85 ± 30.66	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1278840895 \ -pgnout ncm-dbt-20220704-1342-020.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432399	ncm-dbt-01	585885	500	144 108 248	+25.06 ± 14.68	0 38 142 66 4	+44.72 ± 28.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1387108666 \ -pgnout ncm-dbt-20220704-1342-019.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432398	ncm-dbt-05	579909	500	148 113 239	+24.36 ± 13.84	0 35 146 68 1	+47.55 ± 27.73	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 99043291 \ -pgnout ncm-dbt-20220704-1342-018.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432397	ncm-dbt-03	584369	500	132 128 240	+2.78 ± 15.17	1 56 132 60 1	+5.56 ± 29.66	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4141679670 \ -pgnout ncm-dbt-20220704-1342-017.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432396	ncm-dbt-02	583908	500	143 120 237	+15.99 ± 14.38	0 43 143 62 2	+29.25 ± 28.2	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 848658640 \ -pgnout ncm-dbt-20220704-1342-016.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432395	ncm-dbt-04	569789	500	132 126 242	+4.17 ± 15.89	3 56 123 68 0	+12.51 ± 30.78	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1117160132 \ -pgnout ncm-dbt-20220704-1342-015.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432394	ncm-dbt-01	584579	500	143 130 227	+9.04 ± 14.71	2 46 139 63 0	+20.87 ± 28.75	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3683355191 \ -pgnout ncm-dbt-20220704-1342-014.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432393	ncm-dbt-05	583824	500	143 124 233	+13.21 ± 14.28	1 43 142 64 0	+27.85 ± 28.34	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2229233026 \ -pgnout ncm-dbt-20220704-1342-013.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432392	ncm-dbt-03	581361	500	115 120 265	-3.48 ± 14.99	0 62 132 55 1	-8.34 ± 29.66	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 476060573 \ -pgnout ncm-dbt-20220704-1342-012.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432391	ncm-dbt-02	587579	500	143 99 258	+30.65 ± 14.81	0 40 126 84 0	+61.79 ± 30.39	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3481872829 \ -pgnout ncm-dbt-20220704-1342-011.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432390	ncm-dbt-01	583154	500	137 114 249	+15.99 ± 13.99	0 42 143 65 0	+32.05 ± 28.2	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2643630103 \ -pgnout ncm-dbt-20220704-1342-010.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432389	ncm-dbt-04	568434	500	140 124 236	+11.12 ± 14.88	2 46 136 66 0	+25.06 ± 29.14	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1804416481 \ -pgnout ncm-dbt-20220704-1342-009.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432388	ncm-dbt-05	584076	500	144 135 221	+6.26 ± 15.7	1 59 120 70 0	+13.9 ± 31.14	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1452587155 \ -pgnout ncm-dbt-20220704-1342-008.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432387	ncm-dbt-02	587070	500	136 131 233	+3.47 ± 14.09	0 49 149 50 2	+4.17 ± 27.43	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 172176868 \ -pgnout ncm-dbt-20220704-1342-007.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432386	ncm-dbt-03	583950	500	122 127 251	-3.47 ± 15.71	3 60 126 61 0	-2.78 ± 30.41	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 644910012 \ -pgnout ncm-dbt-20220704-1342-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432385	ncm-dbt-04	566967	500	148 112 240	+25.06 ± 14.02	0 36 143 70 1	+48.96 ± 28.14	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3657891736 \ -pgnout ncm-dbt-20220704-1342-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432384	ncm-dbt-01	585000	500	126 107 267	+13.21 ± 14.8	0 49 134 66 1	+25.06 ± 29.4	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4026328931 \ -pgnout ncm-dbt-20220704-1342-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432383	ncm-dbt-02	585717	500	128 117 255	+7.64 ± 14.84	0 52 137 59 2	+12.51 ± 29.02	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 197655791 \ -pgnout ncm-dbt-20220704-1342-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432382	ncm-dbt-03	587707	500	158 117 225	+28.55 ± 14.28	0 37 135 78 0	+57.5 ± 29.2	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3474029731 \ -pgnout ncm-dbt-20220704-1342-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
432381	ncm-dbt-05	580530	500	131 109 260	+15.3 ± 15.45	0 52 126 70 2	+27.85 ± 30.41	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1190532848 \ -pgnout ncm-dbt-20220704-1342-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20220704-1342 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:85f8ee6199f8578fbc082fc0f37e1985813e637a \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15

Commit

Commit ID	85f8ee6199f8578fbc082fc0f37e1985813e637a
Author	Joost VandeVondele
Date	2022-07-04 13:42:34 UTC
Update default net to nn-3c0054ea9860.nnu First things first... this PR is being made from court. Today, Tord and Stéphane, with broad support of the developer community are defending their complaint, filed in Munich, against ChessBase. With their products Houdini 6 and Fat Fritz 2, both Stockfish derivatives, ChessBase violated repeatedly the Stockfish GPLv3 license. Tord and Stéphane have terminated their license with ChessBase permanently. Today we have the opportunity to present our evidence to the judge and enforce that termination. To read up, have a look at our blog post https://stockfishchess.org/blog/2022/public-court-hearing-soon/ and https://stockfishchess.org/blog/2021/our-lawsuit-against-chessbase/ This PR introduces a net trained with an enhanced data set and a modified loss function in the trainer. A slight adjustment for the scaling was needed to get a pass on standard chess. passed STC: https://tests.stockfishchess.org/tests/view/62c0527a49b62510394bd610 LLR: 2.94 (-2.94,2.94) <0.00,2.50> Total: 135008 W: 36614 L: 36152 D: 62242 Ptnml(0-2): 640, 15184, 35407, 15620, 653 passed LTC: https://tests.stockfishchess.org/tests/view/62c17e459e7d9997a12d458e LLR: 2.94 (-2.94,2.94) <0.50,3.00> Total: 28864 W: 8007 L: 7749 D: 13108 Ptnml(0-2): 47, 2810, 8466, 3056, 53 Local testing at a fixed 25k nodes resulted in Test run1026/easy_train_data/experiments/experiment_2/training/run_0/nn-epoch799.nnue localElo: 4.2 +- 1.6 The real strength of the net is in FRC and DFRC chess where it gains significantly. Tested at STC with slightly different scaling: FRC: https://tests.stockfishchess.org/tests/view/62c13a4002ba5d0a774d20d4 Elo: 29.78 +-3.4 (95%) LOS: 100.0% Total: 10000 W: 2007 L: 1152 D: 6841 Ptnml(0-2): 31, 686, 2804, 1355, 124 nElo: 59.24 +-6.9 (95%) PairsRatio: 2.06 DFRC: https://tests.stockfishchess.org/tests/view/62c13a5702ba5d0a774d20d9 Elo: 55.25 +-3.9 (95%) LOS: 100.0% Total: 10000 W: 2984 L: 1407 D: 5609 Ptnml(0-2): 51, 636, 2266, 1779, 268 nElo: 96.95 +-7.2 (95%) PairsRatio: 2.98 Tested at LTC with identical scaling: FRC: https://tests.stockfishchess.org/tests/view/62c26a3c9e7d9997a12d6caf Elo: 16.20 +-2.5 (95%) LOS: 100.0% Total: 10000 W: 1192 L: 726 D: 8082 Ptnml(0-2): 10, 403, 3727, 831, 29 nElo: 44.12 +-6.7 (95%) PairsRatio: 2.08 DFRC: https://tests.stockfishchess.org/tests/view/62c26a539e7d9997a12d6cb2 Elo: 40.94 +-3.0 (95%) LOS: 100.0% Total: 10000 W: 2215 L: 1042 D: 6743 Ptnml(0-2): 10, 410, 3053, 1451, 76 nElo: 92.77 +-6.9 (95%) PairsRatio: 3.64 This is due to the mixing in a significant fraction of DFRC training data in the final training round. The net is trained using the easy_train.py script in the following way: ``` python easy_train.py \ --training-dataset=../Leela-dfrc_n5000.binpack \ --experiment-name=2 \ --nnue-pytorch-branch=vondele/nnue-pytorch/lossScan4 \ --additional-training-arg=--param-index=2 \ --start-lambda=1.0 \ --end-lambda=0.75 \ --gamma=0.995 \ --lr=4.375e-4 \ --start-from-engine-test-net True \ --tui=False \ --seed=$RANDOM \ --max_epoch=800 \ --auto-exit-timeout-on-training-finished=900 \ --network-testing-threads 8 \ --num-workers 12 ``` where the data set used (Leela-dfrc_n5000.binpack) is a combination of our previous best data set (mix of Leela and some SF data) and DFRC data, interleaved to form: The data is available in https://drive.google.com/drive/folders/1S9-ZiQa_3ApmjBtl2e8SyHxj4zG4V8gG?usp=sharing Leela mix: https://drive.google.com/file/d/1JUkMhHSfgIYCjfDNKZUMYZt6L5I7Ra6G/view?usp=sharing DFRC: https://drive.google.com/file/d/17vDaff9LAsVo_1OfsgWAIYqJtqR8aHlm/view?usp=sharing The training branch used is https://github.com/vondele/nnue-pytorch/commits/lossScan4 A PR to the main trainer repo will be made later. This contains a revised loss function, now computing the loss from the score based on the win rate model, which is a more accurate representation than what we had before. Scaling constants are tweaked there as well. closes https://github.com/official-stockfish/Stockfish/pull/4100 Bench: 5186781