Dev Builds » 20230329-1937

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	Wins	Losses	Draws	Elo
ncm-et-3	09:39:55	1948911	3329	2840	4	485	+438.83 +/- 15.46
ncm-et-4	09:39:45	1956874	3344	2855	6	483	+438.92 +/- 15.5
ncm-et-9	09:39:56	1950278	3307	2818	4	485	+437.59 +/- 15.46
ncm-et-10	09:39:30	1950333	3319	2812	8	499	+430.06 +/- 15.25
ncm-et-13	09:39:39	1959423	3378	2889	6	483	+440.82 +/- 15.5
ncm-et-15	09:39:46	1952824	3323	2844	4	475	+442.34 +/- 15.63
			20000	17058	32	2910	+438.07 +/- 6.3

Test Detail

ID	Host	Started (UTC)	Duration	Base NPS	Games	Wins	Losses	Draws	Elo
181596	ncm-et-9	2023-03-31 14:04	00:54:03	1949639	307	266	0	41	+458.14 +/- 54.45
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1946296771 \ -pgnout /tmp/ncm-dbt-20230329-1937-042.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.62+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181595	ncm-et-10	2023-03-31 14:02	00:55:34	1947112	319	263	0	56	+406.69 +/- 46.09
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2849672317 \ -pgnout /tmp/ncm-dbt-20230329-1937-041.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.65+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181594	ncm-et-3	2023-03-31 14:01	00:56:11	1958335	329	282	0	47	+445.57 +/- 50.64
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3488073638 \ -pgnout /tmp/ncm-dbt-20230329-1937-040.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181593	ncm-et-15	2023-03-31 14:01	00:56:58	1958487	323	271	0	52	+423.11 +/- 47.96
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1765654888 \ -pgnout /tmp/ncm-dbt-20230329-1937-039.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181592	ncm-et-4	2023-03-31 13:59	00:59:02	1959438	344	302	0	42	+474.79 +/- 53.81
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3689959402 \ -pgnout /tmp/ncm-dbt-20230329-1937-038.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181591	ncm-et-13	2023-03-31 13:54	01:04:02	1959432	378	320	1	57	+428.95 +/- 45.87
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1092388711 \ -pgnout /tmp/ncm-dbt-20230329-1937-037.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181585	ncm-et-9	2023-03-31 12:36	01:26:45	1956972	500	413	0	87	+408.38 +/- 36.75
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1857004328 \ -pgnout /tmp/ncm-dbt-20230329-1937-036.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181584	ncm-et-3	2023-03-31 12:34	01:26:45	1951152	500	433	0	67	+457.52 +/- 42.15
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 315946603 \ -pgnout /tmp/ncm-dbt-20230329-1937-035.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181583	ncm-et-10	2023-03-31 12:34	01:27:38	1944362	500	426	1	73	+436.43 +/- 40.36
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1721403895 \ -pgnout /tmp/ncm-dbt-20230329-1937-034.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.69+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181582	ncm-et-15	2023-03-31 12:33	01:27:02	1932363	500	437	0	63	+468.95 +/- 43.54
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1804621371 \ -pgnout /tmp/ncm-dbt-20230329-1937-033.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.84+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181581	ncm-et-4	2023-03-31 12:30	01:27:44	1949376	500	429	0	71	+446.7 +/- 40.89
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 816726148 \ -pgnout /tmp/ncm-dbt-20230329-1937-032.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.62+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181580	ncm-et-13	2023-03-31 12:28	01:25:16	1970174	500	436	1	63	+463.16 +/- 43.6
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2469742949 \ -pgnout /tmp/ncm-dbt-20230329-1937-031.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.36+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181574	ncm-et-9	2023-03-31 11:08	01:27:37	1952548	500	431	1	68	+449.35 +/- 41.89
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1516337961 \ -pgnout /tmp/ncm-dbt-20230329-1937-030.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.58+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181573	ncm-et-3	2023-03-31 11:07	01:26:03	1956057	500	419	0	81	+421.93 +/- 38.15
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1286485826 \ -pgnout /tmp/ncm-dbt-20230329-1937-029.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.54+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181572	ncm-et-10	2023-03-31 11:07	01:26:01	1964672	500	422	2	76	+424.28 +/- 39.56
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3324371750 \ -pgnout /tmp/ncm-dbt-20230329-1937-028.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.43+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181571	ncm-et-15	2023-03-31 11:05	01:27:36	1947897	500	427	0	73	+441.5 +/- 40.29
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1828083745 \ -pgnout /tmp/ncm-dbt-20230329-1937-027.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.64+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181570	ncm-et-4	2023-03-31 11:04	01:25:41	1967544	500	435	3	62	+454.76 +/- 43.98
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2937749520 \ -pgnout /tmp/ncm-dbt-20230329-1937-026.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.40+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181569	ncm-et-13	2023-03-31 11:01	01:25:42	1971732	500	430	1	69	+446.7 +/- 41.57
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2246966708 \ -pgnout /tmp/ncm-dbt-20230329-1937-025.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.34+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181560	ncm-et-9	2023-03-31 09:39	01:27:43	1957889	500	430	0	70	+449.35 +/- 41.19
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1866989809 \ -pgnout /tmp/ncm-dbt-20230329-1937-024.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181559	ncm-et-3	2023-03-31 09:39	01:27:25	1951461	500	441	1	58	+477.99 +/- 45.54
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1201949461 \ -pgnout /tmp/ncm-dbt-20230329-1937-023.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181558	ncm-et-10	2023-03-31 09:38	01:28:20	1954505	500	420	1	79	+421.93 +/- 38.73
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2359949377 \ -pgnout /tmp/ncm-dbt-20230329-1937-022.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.56+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181557	ncm-et-4	2023-03-31 09:37	01:26:24	1954371	500	419	0	81	+421.93 +/- 38.15
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3985873552 \ -pgnout /tmp/ncm-dbt-20230329-1937-021.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.56+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181556	ncm-et-15	2023-03-31 09:36	01:27:42	1956647	500	437	0	63	+468.95 +/- 43.54
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3562091910 \ -pgnout /tmp/ncm-dbt-20230329-1937-020.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181555	ncm-et-13	2023-03-31 09:34	01:26:08	1948748	500	422	0	78	+429.05 +/- 38.92
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3629549414 \ -pgnout /tmp/ncm-dbt-20230329-1937-019.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.63+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181549	ncm-et-9	2023-03-31 08:12	01:26:37	1956660	500	435	2	63	+457.52 +/- 43.63
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3837692453 \ -pgnout /tmp/ncm-dbt-20230329-1937-018.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181548	ncm-et-3	2023-03-31 08:11	01:27:11	1947538	500	424	0	76	+433.94 +/- 39.45
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1080618310 \ -pgnout /tmp/ncm-dbt-20230329-1937-017.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.65+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181547	ncm-et-10	2023-03-31 08:10	01:27:39	1946004	500	439	0	61	+474.93 +/- 44.28
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 696036048 \ -pgnout /tmp/ncm-dbt-20230329-1937-016.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.67+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181546	ncm-et-4	2023-03-31 08:09	01:27:29	1950102	500	414	1	85	+408.38 +/- 37.27
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3609935815 \ -pgnout /tmp/ncm-dbt-20230329-1937-015.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.61+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181545	ncm-et-15	2023-03-31 08:09	01:27:12	1951018	500	421	2	77	+421.93 +/- 39.29
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2019043833 \ -pgnout /tmp/ncm-dbt-20230329-1937-014.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181544	ncm-et-13	2023-03-31 08:07	01:26:54	1951154	500	413	3	84	+401.92 +/- 37.57
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 666052426 \ -pgnout /tmp/ncm-dbt-20230329-1937-013.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181538	ncm-et-9	2023-03-31 06:44	01:27:58	1938077	500	416	0	84	+415.04 +/- 37.43
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1614282734 \ -pgnout /tmp/ncm-dbt-20230329-1937-012.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.77+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181537	ncm-et-3	2023-03-31 06:42	01:28:51	1936697	500	419	0	81	+421.93 +/- 38.15
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2607782873 \ -pgnout /tmp/ncm-dbt-20230329-1937-011.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.78+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181536	ncm-et-10	2023-03-31 06:41	01:27:46	1951608	500	428	2	70	+438.95 +/- 41.29
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 4259400620 \ -pgnout /tmp/ncm-dbt-20230329-1937-010.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181535	ncm-et-4	2023-03-31 06:41	01:26:55	1958964	500	434	2	64	+454.76 +/- 43.27
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3494062773 \ -pgnout /tmp/ncm-dbt-20230329-1937-009.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181534	ncm-et-15	2023-03-31 06:41	01:27:04	1957514	500	423	2	75	+426.65 +/- 39.83
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2700816021 \ -pgnout /tmp/ncm-dbt-20230329-1937-008.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181533	ncm-et-13	2023-03-31 06:39	01:27:03	1953487	500	440	0	60	+477.98 +/- 44.67
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1731615972 \ -pgnout /tmp/ncm-dbt-20230329-1937-007.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.57+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181527	ncm-et-10	2023-03-31 05:14	01:26:32	1944069	500	414	2	84	+406.2 +/- 37.55
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 1159640484 \ -pgnout /tmp/ncm-dbt-20230329-1937-006.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.69+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181526	ncm-et-13	2023-03-31 05:14	01:24:34	1961239	500	428	0	72	+444.08 +/- 40.59
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2516626919 \ -pgnout /tmp/ncm-dbt-20230329-1937-005.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.47+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181525	ncm-et-15	2023-03-31 05:14	01:26:12	1965844	500	428	0	72	+444.08 +/- 40.59
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 3225947528 \ -pgnout /tmp/ncm-dbt-20230329-1937-004.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.42+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181524	ncm-et-4	2023-03-31 05:14	01:26:30	1958326	500	422	0	78	+429.05 +/- 38.92
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2784435309 \ -pgnout /tmp/ncm-dbt-20230329-1937-003.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181523	ncm-et-3	2023-03-31 05:14	01:27:29	1941139	500	422	3	75	+421.93 +/- 39.85
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 923469835 \ -pgnout /tmp/ncm-dbt-20230329-1937-002.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.73+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
181522	ncm-et-9	2023-03-31 05:14	01:29:13	1940167	500	427	1	72	+438.95 +/- 40.66
cutechess-cli \ -debug \ -games 500 \ -concurrency 7 \ -srand 2630262443 \ -pgnout /tmp/ncm-dbt-20230329-1937-001.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.74+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20230329-1937 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:37160c4b1632245d46d86cec7bd22b76f5a87531 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7

Commit

Commit ID	37160c4b1632245d46d86cec7bd22b76f5a87531
Author	Linmiao Xu
Date	2023-03-29 19:37:52 UTC
Update default net to nn-dabb1ed23026.nnue Created by retraining the master net with these modifications: * New filtering methods for existing data from T80 sep+oct2022, T79 apr2022, T78 jun+jul+aug+sep2022, T77 dec2021 * Adding new filtered data from T80 aug2022 and T78 apr+may2022 * Increasing early-fen-skipping from 28 to 30 ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3-sk30 \ --training-dataset /data/leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --max_epoch 900 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The v3 filtering used for data from T77dec 2021 differs from v2 filtering in that: * To improve binpack compression, positions after ply 28 were skipped during training by setting position scores to VALUE_NONE (32002) instead of removing them entirely * All early-game positions with ply <= 28 were removed to maximize binpack compression * Only bestmove captures at d6pv2 search were skipped, not 2nd bestmove captures * Binpack compression was repaired for the remaining positions by effectively replacing bestmoves with "played moves" to maintain contiguous sequences of positions in the training game data After improving binpack compression, The T77 dec2021 data size was reduced from 95G to 19G. The v6 filtering used for data from T80augsepoctT79aprT78aprtosep 2022 differs from v2 in that: * All positions with only one legal move were removed * Tighter score differences at d6pv2 search were used to remove more positions with only one good move than before * d6pv2 search was not used to remove positions where the best 2 moves were captures ``` python3 interleave_binpacks.py \ nn-547-dataset/leela96-eval-filt-v2.binpack \ nn-547-dataset/dfrc99-eval-filt-v2.binpack \ nn-547-dataset/test80-nov2022-12tb7p-eval-filt-v2-d6.binpack \ nn-547-dataset/T79-may2022-12tb7p-eval-filt-v2.binpack \ nn-547-dataset/T60-nov2021-12tb7p-eval-filt-v2.binpack \ nn-547-dataset/T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.binpack \ filt-v6/test80-oct2022-16tb7p-filter-v6.binpack \ filt-v6/test79-apr2022-16tb7p-filter-v6.binpack \ filt-v6/test78-aprmay2022-16tb7p-filter-v6.binpack \ filt-v6/test78-junjulaug2022-16tb7p-filter-v6.binpack \ filt-v6/test78-sep2022-16tb7p-filter-v6.binpack \ filt-v3/test77-dec2021-16tb7p-filt-v3.binpack \ /data/leela96-dfrc99-T80novT79mayT60novdec-v2-T80augsepoctT79aprT78aprtosep-v6-T77dec-v3.binpack ``` The code for the new data filtering methods is available at: https://github.com/linrock/Stockfish/tree/nnue-data-v3/nnue-data The code for giving hexword names to .nnue files is at: https://github.com/linrock/nnue-namer Links for downloading the training data components can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch779.nnue : 0.6 +/- 3.1 Passed STC: https://tests.stockfishchess.org/tests/view/64212412db43ab2ba6f8efb0 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 82256 W: 22185 L: 21809 D: 38262 Ptnml(0-2): 286, 9065, 22067, 9407, 303 Passed LTC: https://tests.stockfishchess.org/tests/view/64223726db43ab2ba6f91d6c LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 30840 W: 8437 L: 8149 D: 14254 Ptnml(0-2): 14, 2891, 9323, 3177, 15 closes https://github.com/official-stockfish/Stockfish/pull/4465 bench 5101970