Dev Builds » 20230701-1134

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo
ncm-dbt-01	06:53:39	584084	4000	1316 769 1915	+47.81 ± 5.17	3 226 1002 759 10	+96.19 ± 10.73
ncm-dbt-02	06:54:19	585165	4000	1380 736 1884	+56.43 ± 5.13	2 197 966 825 10	+114.45 ± 10.93
ncm-dbt-03	06:55:12	586406	4000	1348 735 1917	+53.67 ± 5.1	3 197 993 798 9	+108.88 ± 10.76
ncm-dbt-04	06:53:15	570574	4000	1320 741 1939	+50.65 ± 5.04	4 196 1024 769 7	+102.97 ± 10.57
ncm-dbt-05	06:54:25	584732	4000	1350 708 1942	+56.25 ± 5.12	3 195 967 827 8	+114.65 ± 10.92
			20000	6714 3689 9597	+52.96 ± 2.29	15 1011 4952 3978 44	+107.39 ± 4.82

Test Detail

ID	Host	Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo	PGN
393508	ncm-dbt-01	582611	500	150 101 249	+34.16 ± 14.26	0 32 139 77 2	+66.1 ± 28.61	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1778423582 \ -pgnout ncm-dbt-20230701-1134-040.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393507	ncm-dbt-03	586774	500	174 100 226	+51.8 ± 14.49	0 26 126 96 2	+102.97 ± 30.28	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2257991550 \ -pgnout ncm-dbt-20230701-1134-039.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393506	ncm-dbt-05	582235	500	164 86 250	+54.65 ± 14.49	1 24 121 104 0	+113.68 ± 30.97	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2040284932 \ -pgnout ncm-dbt-20230701-1134-038.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393505	ncm-dbt-04	571431	500	175 104 221	+49.67 ± 13.69	1 20 136 93 0	+102.97 ± 28.77	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2984836657 \ -pgnout ncm-dbt-20230701-1134-037.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393504	ncm-dbt-02	585379	500	165 97 238	+47.55 ± 14.66	1 25 132 89 3	+93.95 ± 29.45	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2104192979 \ -pgnout ncm-dbt-20230701-1134-036.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393503	ncm-dbt-03	587919	500	170 93 237	+53.93 ± 14.73	0 28 118 103 1	+109.07 ± 31.43	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 184447260 \ -pgnout ncm-dbt-20230701-1134-035.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393502	ncm-dbt-05	579413	500	165 92 243	+51.09 ± 14.31	0 27 123 100 0	+104.49 ± 30.71	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2272613537 \ -pgnout ncm-dbt-20230701-1134-034.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393501	ncm-dbt-01	581985	500	159 99 242	+41.89 ± 14.61	1 29 130 89 1	+85.04 ± 29.79	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 236312000 \ -pgnout ncm-dbt-20230701-1134-033.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393500	ncm-dbt-04	572195	500	154 82 264	+50.38 ± 13.72	0 23 132 95 0	+102.97 ± 29.38	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1251384413 \ -pgnout ncm-dbt-20230701-1134-032.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393499	ncm-dbt-02	587197	500	171 91 238	+56.07 ± 13.87	0 22 126 102 0	+115.23 ± 30.2	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 159296838 \ -pgnout ncm-dbt-20230701-1134-031.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393498	ncm-dbt-03	583908	500	162 91 247	+49.67 ± 15.16	0 33 114 102 1	+99.95 ± 32.0	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4035495473 \ -pgnout ncm-dbt-20230701-1134-030.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393497	ncm-dbt-05	582611	500	170 87 243	+58.21 ± 15.06	1 25 116 106 2	+118.33 ± 31.7	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1187733890 \ -pgnout ncm-dbt-20230701-1134-029.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393496	ncm-dbt-02	585211	500	177 90 233	+61.07 ± 14.92	0 26 113 109 2	+123.02 ± 32.15	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 546264361 \ -pgnout ncm-dbt-20230701-1134-028.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393495	ncm-dbt-04	570469	500	164 84 252	+56.07 ± 14.83	1 25 118 105 1	+115.23 ± 31.41	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3710620841 \ -pgnout ncm-dbt-20230701-1134-027.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393494	ncm-dbt-01	586901	500	164 90 246	+51.8 ± 14.08	0 23 132 93 2	+102.97 ± 29.38	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 687609549 \ -pgnout ncm-dbt-20230701-1134-026.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393493	ncm-dbt-03	585717	500	175 85 240	+63.23 ± 14.06	0 20 121 108 1	+129.35 ± 30.88	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2041097805 \ -pgnout ncm-dbt-20230701-1134-025.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393492	ncm-dbt-04	569949	500	168 88 244	+56.07 ± 14.43	0 25 121 103 1	+113.68 ± 30.97	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2179322574 \ -pgnout ncm-dbt-20230701-1134-024.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393491	ncm-dbt-05	583656	500	166 100 234	+46.13 ± 14.58	1 27 128 93 1	+93.95 ± 30.03	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 382350197 \ -pgnout ncm-dbt-20230701-1134-023.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393490	ncm-dbt-02	585759	500	173 103 224	+48.96 ± 14.07	1 23 131 95 0	+101.46 ± 29.54	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2671258902 \ -pgnout ncm-dbt-20230701-1134-022.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393489	ncm-dbt-01	584285	500	171 89 240	+57.5 ± 14.76	1 25 115 109 0	+119.89 ± 31.85	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4137610942 \ -pgnout ncm-dbt-20230701-1134-021.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393488	ncm-dbt-04	570068	500	172 101 227	+49.67 ± 13.97	0 24 132 93 1	+99.95 ± 29.4	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3763386334 \ -pgnout ncm-dbt-20230701-1134-020.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393487	ncm-dbt-01	581652	500	175 88 237	+61.07 ± 14.52	0 24 116 109 1	+124.6 ± 31.69	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 364383650 \ -pgnout ncm-dbt-20230701-1134-019.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393486	ncm-dbt-05	585928	500	164 83 253	+56.78 ± 14.05	0 20 132 95 3	+112.14 ± 29.29	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1674564957 \ -pgnout ncm-dbt-20230701-1134-018.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393485	ncm-dbt-03	586520	500	163 84 253	+55.36 ± 14.39	2 19 128 100 1	+115.23 ± 29.89	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1216992319 \ -pgnout ncm-dbt-20230701-1134-017.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393484	ncm-dbt-02	581693	500	175 91 234	+58.93 ± 14.83	0 27 113 109 1	+119.89 ± 32.15	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3476850330 \ -pgnout ncm-dbt-20230701-1134-016.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393483	ncm-dbt-04	569509	500	158 93 249	+45.42 ± 14.81	1 30 122 97 0	+93.95 ± 30.89	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3048621861 \ -pgnout ncm-dbt-20230701-1134-015.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393482	ncm-dbt-02	583950	500	179 80 241	+69.71 ± 14.31	0 20 112 117 1	+143.89 ± 32.25	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 498412563 \ -pgnout ncm-dbt-20230701-1134-014.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393481	ncm-dbt-01	582903	500	165 102 233	+44.01 ± 14.99	0 34 120 95 1	+88.0 ± 31.17	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2304155777 \ -pgnout ncm-dbt-20230701-1134-013.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393480	ncm-dbt-05	584201	500	171 85 244	+60.36 ± 13.79	0 19 127 103 1	+123.02 ± 29.98	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 4110762025 \ -pgnout ncm-dbt-20230701-1134-012.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393479	ncm-dbt-03	586943	500	167 105 228	+43.3 ± 13.17	0 22 144 84 0	+88.0 ± 27.7	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 698967932 \ -pgnout ncm-dbt-20230701-1134-011.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393478	ncm-dbt-03	585590	500	166 84 250	+57.5 ± 14.36	0 23 124 101 2	+115.23 ± 30.51	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2264886094 \ -pgnout ncm-dbt-20230701-1134-010.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393477	ncm-dbt-02	583405	500	170 85 245	+59.64 ± 14.18	0 22 122 105 1	+121.46 ± 30.78	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3497003650 \ -pgnout ncm-dbt-20230701-1134-009.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393476	ncm-dbt-04	570388	500	161 93 246	+47.55 ± 13.99	1 21 139 87 2	+95.44 ± 28.38	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3417991847 \ -pgnout ncm-dbt-20230701-1134-008.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393475	ncm-dbt-05	588387	500	172 82 246	+63.23 ± 14.88	0 27 106 117 0	+130.94 ± 33.2	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1717847488 \ -pgnout ncm-dbt-20230701-1134-007.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393474	ncm-dbt-01	583992	500	167 94 239	+51.09 ± 14.45	1 23 130 94 2	+102.97 ± 29.68	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 2496430176 \ -pgnout ncm-dbt-20230701-1134-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393473	ncm-dbt-04	570589	500	168 96 236	+50.38 ± 14.68	0 28 124 96 2	+99.95 ± 30.58	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3841493379 \ -pgnout ncm-dbt-20230701-1134-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393472	ncm-dbt-01	588345	500	165 106 229	+41.19 ± 15.08	0 36 120 93 1	+82.1 ± 31.18	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1567558164 \ -pgnout ncm-dbt-20230701-1134-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393471	ncm-dbt-05	591426	500	178 93 229	+59.64 ± 14.73	0 26 114 109 1	+121.46 ± 32.0	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 3092518293 \ -pgnout ncm-dbt-20230701-1134-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393470	ncm-dbt-03	587877	500	171 93 236	+54.65 ± 14.89	1 26 118 104 1	+112.14 ± 31.42	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 773847311 \ -pgnout ncm-dbt-20230701-1134-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15
393469	ncm-dbt-02	588728	500	170 99 231	+49.67 ± 15.16	0 32 117 99 2	+98.44 ± 31.58	↓
cutechess-cli \ -rounds 266 \ -games 2 \ -concurrency 16 \ -srand 1090367200 \ -pgnout ncm-dbt-20230701-1134-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=8 \ -engine \ name=20230701-1134 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:915532181f11812c80ef0b57bc018de4ea2155ec \ -engine \ name=sf15 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:15

Commit

Commit ID	915532181f11812c80ef0b57bc018de4ea2155ec
Author	Linmiao Xu
Date	2023-07-01 11:34:30 UTC
Update NNUE architecture to SFNNv7 with larger L1 size of 2048 Creating this net involved: - a 5-step training process from scratch - greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620 - leb128 compression with https://github.com/glinscott/nnue-pytorch/pull/251 - greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640 The 5 training steps were: 1. 400 epochs, lambda 1.0, lr 9.75e-4 UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9.binpack (178G) nodes5000pv2_UHO.binpack data_pv-2_diff-100_nodes-5000.binpack wrongIsRight_nodes5000pv2.binpack multinet_pv-2_diff-100_nodes-5000.binpack dfrc_n5000.binpack large_gensfen_multipvdiff_100_d9.binpack ep399 chosen as start model for step2 2. 800 epochs, end-lambda 0.75, skip 16 LeelaFarseer-T78juntoaugT79marT80dec.binpack (141G) T60T70wIsRightFarseerT60T74T75T76.binpack test78-junjulaug2022-16tb7p.no-db.min.binpack test79-mar2022-16tb7p.no-db.min.binpack test80-dec2022-16tb7p.no-db.min.binpack ep559 chosen as start model for step3 3. 800 epochs, end-lambda 0.725, skip 20 leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr.binpack (223G) leela96-filt-v2.min.binpack dfrc99-16tb7p-eval-filt-v2.min.binpack test80-dec2022-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-jan2023-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-feb2023-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-mar2023-2tb7p-filter-v6.min.binpack test77-dec2021-16tb7p.no-db.min.binpack test78-janfeb2022-16tb7p.no-db.min.binpack test79-apr2022-16tb7p.no-db.min.binpack ep499 chosen as start model for step4 4. 800 epochs, end-lambda 0.7, skip 24 0dd1cebea57 dataset https://github.com/official-stockfish/Stockfish/pull/4606 ep599 chosen as start model for step5 5. 800 epochs, end-lambda 0.7, skip 28 same dataset as step4 ep619 became nn-1b951f8b449d.nnue For the final step5 training: python3 easy_train.py \ --experiment-name L1-2048-S5-sameData-sk28-S4-0dd1cebea57-shuffled-S3-leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr-sk20-S2-LeelaFarseerT78T79T80-ep399-S1-UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --early-fen-skipping 28 \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-2048 \ --engine-test-branch linrock/Stockfish/L1-2048 \ --start-from-engine-test-net False \ --start-from-model /data/experiments/experiment_L1-2048-S4-0dd1cebea57-shuffled-S3-leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr-sk20-S2-LeelaFarseerT78T79T80-ep399-S1-UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9/training/run_0/nn-epoch599.nnue --max_epoch 800 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 SF training data components for the step1 dataset: https://drive.google.com/drive/folders/1yLCEmioC3Xx9KQr4T7uB6GnLm5icAYGU Leela training data for steps 2-5 can be found at: https://robotmoon.com/nnue-training-data/ Due to larger L1 size and slower inference, the speed penalty loses elo at STC. Measurements from 100 bench runs at depth 13 with x86-64-modern on Intel Core i5-1038NG7 2.00GHz: sf_base = 1240730 +/- 3443 (95%) sf_test = 1153341 +/- 2832 (95%) diff = -87388 +/- 1616 (95%) speedup = -7.04330% +/- 0.130% (95%) Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue): nn-epoch619.nnue : 21.1 +/- 3.2 Failed STC: https://tests.stockfishchess.org/tests/view/6498ee93dc7002ce609cf979 LLR: -2.95 (-2.94,2.94) <0.00,2.00> Total: 11680 W: 3058 L: 3299 D: 5323 Ptnml(0-2): 44, 1422, 3149, 1181, 44 LTC: https://tests.stockfishchess.org/tests/view/649b32f5dc7002ce609d20cf Elo: 0.68 ± 1.5 (95%) LOS: 80.5% Total: 40000 W: 10887 L: 10809 D: 18304 Ptnml(0-2): 36, 3938, 11958, 4048, 20 nElo: 1.50 ± 3.4 (95%) PairsRatio: 1.02 Passed VLTC 180+1.8: https://tests.stockfishchess.org/tests/view/64992b43dc7002ce609cfd20 LLR: 3.06 (-2.94,2.94) <0.00,2.00> Total: 38086 W: 10612 L: 10338 D: 17136 Ptnml(0-2): 9, 3316, 12115, 3598, 5 Passed VLTC SMP 60+0.6 th 8: https://tests.stockfishchess.org/tests/view/649a21fedc7002ce609d0c7d LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 38936 W: 11091 L: 10820 D: 17025 Ptnml(0-2): 1, 2948, 13305, 3207, 7 closes https://github.com/official-stockfish/Stockfish/pull/4646 Bench: 2505168