Dev Builds » 20221119-0829

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	Wins	Losses	Draws	Elo
ncm-et-3	11:18:38	1921934	3290	2815	4	471	+442.03 ± 15.69
ncm-et-4	11:18:13	1928765	3338	2845	4	489	+437.82 ± 15.4
ncm-et-9	11:19:01	1953618	3348	2863	6	479	+440.66 ± 15.57
ncm-et-10	11:19:26	1956923	3381	2905	5	471	+446.35 ± 15.7
ncm-et-13	11:18:14	1920114	3288	2781	3	504	+430.13 ± 15.16
ncm-et-15	11:19:26	1951842	3355	2864	4	487	+439.53 ± 15.43
			20000	17073	26	2901	+439.4 ± 6.31

Test Detail

ID	Host	Started (UTC)	Duration	Base NPS	Games	Wins	Losses	Draws	Elo	PGN
162090	ncm-et-13	2022-11-22 19:02	00:02:24	1955781	8	8	0	0	+1199.83 ± 190.04	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 795429997 \ -pgnout /tmp/ncm-dbt-20221119-0829-045.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.54+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162089	ncm-et-3	2022-11-22 19:01	00:04:23	1960158	19	16	0	3	+426.66 ± 456.54	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 4232489631 \ -pgnout /tmp/ncm-dbt-20221119-0829-044.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.49+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162088	ncm-et-4	2022-11-22 18:53	00:11:49	1953995	57	47	0	10	+406.78 ± 120.06	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 4093100838 \ -pgnout /tmp/ncm-dbt-20221119-0829-043.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.57+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162087	ncm-et-9	2022-11-22 17:56	01:08:43	1962002	348	293	1	54	+423.2 ± 47.16	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3575241541 \ -pgnout /tmp/ncm-dbt-20221119-0829-042.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.46+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162086	ncm-et-15	2022-11-22 17:52	01:12:36	1942388	355	306	0	49	+452.0 ± 49.57	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2941320629 \ -pgnout /tmp/ncm-dbt-20221119-0829-041.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.71+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162085	ncm-et-10	2022-11-22 17:47	01:17:23	1952521	381	325	1	55	+436.93 ± 46.74	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 65447650 \ -pgnout /tmp/ncm-dbt-20221119-0829-040.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.58+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162084	ncm-et-13	2022-11-22 17:23	01:38:40	1961510	500	425	0	75	+436.43 ± 39.73	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1398666442 \ -pgnout /tmp/ncm-dbt-20221119-0829-039.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.47+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162083	ncm-et-3	2022-11-22 17:20	01:40:06	1958776	500	426	2	72	+433.94 ± 40.69	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2592842805 \ -pgnout /tmp/ncm-dbt-20221119-0829-038.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162082	ncm-et-4	2022-11-22 17:13	01:39:18	1964807	500	422	1	77	+426.65 ± 39.25	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3875286392 \ -pgnout /tmp/ncm-dbt-20221119-0829-037.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.43+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162081	ncm-et-9	2022-11-22 16:12	01:42:53	1938843	500	422	0	78	+429.05 ± 38.92	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2791512858 \ -pgnout /tmp/ncm-dbt-20221119-0829-036.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.76+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162080	ncm-et-15	2022-11-22 16:12	01:39:39	1960922	500	421	2	77	+421.93 ± 39.29	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3746790587 \ -pgnout /tmp/ncm-dbt-20221119-0829-035.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.48+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162079	ncm-et-10	2022-11-22 16:05	01:41:33	1947853	500	441	0	59	+481.09 ± 45.07	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 4146709868 \ -pgnout /tmp/ncm-dbt-20221119-0829-034.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.64+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162078	ncm-et-13	2022-11-22 15:40	01:41:50	1939728	500	411	0	89	+404.05 ± 36.31	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2376887179 \ -pgnout /tmp/ncm-dbt-20221119-0829-033.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.75+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162077	ncm-et-3	2022-11-22 15:39	01:40:13	1962613	500	435	0	65	+463.15 ± 42.83	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 4122950336 \ -pgnout /tmp/ncm-dbt-20221119-0829-032.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.46+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162076	ncm-et-4	2022-11-22 15:34	01:38:36	1966500	500	416	1	83	+412.8 ± 37.74	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1195942420 \ -pgnout /tmp/ncm-dbt-20221119-0829-031.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.41+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162075	ncm-et-15	2022-11-22 14:31	01:40:01	1950849	500	417	0	83	+417.31 ± 37.67	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3912378249 \ -pgnout /tmp/ncm-dbt-20221119-0829-030.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162074	ncm-et-9	2022-11-22 14:31	01:41:09	1956197	500	450	0	50	+511.49 ± 49.18	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1868407627 \ -pgnout /tmp/ncm-dbt-20221119-0829-029.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.54+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162073	ncm-et-10	2022-11-22 14:25	01:39:32	1957421	500	429	2	69	+441.5 ± 41.61	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1600961406 \ -pgnout /tmp/ncm-dbt-20221119-0829-028.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162072	ncm-et-13	2022-11-22 13:59	01:40:50	1956183	500	419	1	80	+419.61 ± 38.47	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2906809198 \ -pgnout /tmp/ncm-dbt-20221119-0829-027.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.54+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162071	ncm-et-3	2022-11-22 13:56	01:42:02	1957279	500	434	0	66	+460.32 ± 42.48	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3723757000 \ -pgnout /tmp/ncm-dbt-20221119-0829-026.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162070	ncm-et-4	2022-11-22 13:53	01:40:46	1956814	500	426	0	74	+438.95 ± 40.01	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3297468572 \ -pgnout /tmp/ncm-dbt-20221119-0829-025.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162069	ncm-et-9	2022-11-22 12:49	01:41:29	1956640	500	427	1	72	+438.95 ± 40.66	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3907466413 \ -pgnout /tmp/ncm-dbt-20221119-0829-024.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162068	ncm-et-15	2022-11-22 12:48	01:42:47	1941614	500	426	1	73	+436.43 ± 40.36	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3502594556 \ -pgnout /tmp/ncm-dbt-20221119-0829-023.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.72+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162067	ncm-et-10	2022-11-22 12:44	01:40:52	1958636	500	432	0	68	+454.76 ± 41.82	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3773649525 \ -pgnout /tmp/ncm-dbt-20221119-0829-022.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162066	ncm-et-13	2022-11-22 12:17	01:40:54	1954683	500	431	1	68	+449.35 ± 41.89	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2136219261 \ -pgnout /tmp/ncm-dbt-20221119-0829-021.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.56+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162065	ncm-et-3	2022-11-22 12:14	01:41:39	1926111	500	422	0	78	+429.05 ± 38.92	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 797053067 \ -pgnout /tmp/ncm-dbt-20221119-0829-020.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.92+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162064	ncm-et-4	2022-11-22 12:13	01:39:15	1965280	500	431	1	68	+449.35 ± 41.89	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2704075478 \ -pgnout /tmp/ncm-dbt-20221119-0829-019.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.42+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162063	ncm-et-15	2022-11-22 11:07	01:40:20	1960775	500	431	1	68	+449.35 ± 41.89	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1518147805 \ -pgnout /tmp/ncm-dbt-20221119-0829-018.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.48+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162062	ncm-et-9	2022-11-22 11:07	01:41:01	1944659	500	423	1	76	+429.05 ± 39.52	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 167644336 \ -pgnout /tmp/ncm-dbt-20221119-0829-017.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.68+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162061	ncm-et-10	2022-11-22 11:04	01:39:03	1965379	500	434	1	65	+457.52 ± 42.89	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 879864845 \ -pgnout /tmp/ncm-dbt-20221119-0829-016.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.42+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162060	ncm-et-4	2022-11-22 10:18	01:54:01	1703238	500	432	0	68	+454.76 ± 41.82	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1724020203 \ -pgnout /tmp/ncm-dbt-20221119-0829-015.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=28.18+0.28 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162059	ncm-et-13	2022-11-22 10:18	01:58:58	1666557	500	424	1	75	+431.48 ± 39.8	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3473370943 \ -pgnout /tmp/ncm-dbt-20221119-0829-014.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=28.80+0.29 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162058	ncm-et-3	2022-11-22 10:18	01:55:45	1691287	500	431	2	67	+446.7 ± 42.25	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3045234936 \ -pgnout /tmp/ncm-dbt-20221119-0829-013.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=28.38+0.28 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162057	ncm-et-9	2022-11-22 09:24	01:41:57	1959102	500	428	1	71	+441.5 ± 40.95	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1455934247 \ -pgnout /tmp/ncm-dbt-20221119-0829-012.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162056	ncm-et-15	2022-11-22 09:24	01:42:03	1955156	500	433	0	67	+457.52 ± 42.15	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2176361874 \ -pgnout /tmp/ncm-dbt-20221119-0829-011.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.55+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162055	ncm-et-3	2022-11-22 09:22	00:54:43	1958628	271	230	0	41	+434.81 ± 54.36
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2799271364 \ -pgnout /tmp/ncm-dbt-20221119-0829-010.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162054	ncm-et-10	2022-11-22 09:22	01:41:42	1957263	500	422	1	77	+426.65 ± 39.25	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1756659532 \ -pgnout /tmp/ncm-dbt-20221119-0829-009.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162053	ncm-et-4	2022-11-22 09:21	00:55:46	1959432	281	240	0	41	+441.61 ± 54.39
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1712922521 \ -pgnout /tmp/ncm-dbt-20221119-0829-008.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162052	ncm-et-13	2022-11-22 09:21	00:55:53	1960931	280	243	0	37	+460.11 ± 57.49
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 20780854 \ -pgnout /tmp/ncm-dbt-20221119-0829-007.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.48+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162051	ncm-et-9	2022-11-22 07:42	01:41:49	1957888	500	420	2	78	+419.61 ± 39.03	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 2485547430 \ -pgnout /tmp/ncm-dbt-20221119-0829-006.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.52+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162050	ncm-et-3	2022-11-22 07:42	01:39:47	1960622	500	421	0	79	+426.65 ± 38.66	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1728256011 \ -pgnout /tmp/ncm-dbt-20221119-0829-005.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.48+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162049	ncm-et-4	2022-11-22 07:42	01:38:42	1960059	500	431	1	68	+449.35 ± 41.89	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3817046327 \ -pgnout /tmp/ncm-dbt-20221119-0829-004.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.49+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162048	ncm-et-10	2022-11-22 07:42	01:39:21	1959389	500	422	0	78	+429.05 ± 38.92	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1875457001 \ -pgnout /tmp/ncm-dbt-20221119-0829-003.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162047	ncm-et-15	2022-11-22 07:42	01:42:00	1951196	500	430	0	70	+449.35 ± 41.19	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 3611083790 \ -pgnout /tmp/ncm-dbt-20221119-0829-002.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.60+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7
162046	ncm-et-13	2022-11-22 07:42	01:38:45	1965540	500	420	0	80	+424.27 ± 38.4	↓
cutechess-cli \ -debug \ -games 500 \ -concurrency 6 \ -srand 1029241827 \ -pgnout /tmp/ncm-dbt-20221119-0829-001.pgn \ -openings \ file=8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.42+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=32 \ option.Threads=2 \ -engine \ name=20221119-0829 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:41c6a74d372799c6ed94b842db66f956d080e0b0 \ -engine \ name=sf7 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:7

Commit

Commit ID	41c6a74d372799c6ed94b842db66f956d080e0b0
Author	VoyagerOne
Date	2022-11-19 08:29:04 UTC
Simplification away Cutoff Reset STC: LLR: 2.95 (-2.94,2.94) <-1.75,0.25> Total: 150184 W: 39913 L: 39819 D: 70452 Ptnml(0-2): 493, 16796, 40474, 16782, 547 https://tests.stockfishchess.org/tests/view/63723e9e54d69a2f33911d3c LTC: LLR: 2.94 (-2.94,2.94) <-1.75,0.25> Total: 58880 W: 15890 L: 15717 D: 27273 Ptnml(0-2): 35, 5765, 17659, 5954, 27 https://tests.stockfishchess.org/tests/view/6373baf49849fa7a36a65427 closes https://github.com/official-stockfish/Stockfish/pull/4231 Bench: 4035152