Dev Builds » 20230629-0600

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo
ncm-dbt-01	09:58:46	1199809	3346	1436 294 1616	+123.54 ± 5.34	0 49 471 1115 38	+275.39 ± 15.71
ncm-dbt-02	09:51:48	1227866	3310	1396 284 1630	+121.44 ± 5.29	0 37 510 1067 41	+267.65 ± 15.07
ncm-dbt-03	09:51:19	1238706	3312	1427 260 1625	+127.9 ± 5.22	0 32 468 1113 43	+287.25 ± 15.75
ncm-dbt-04	09:58:22	1221442	3360	1434 284 1642	+123.91 ± 5.34	0 44 487 1104 45	+274.06 ± 15.45
ncm-dbt-05	09:51:08	1231404	3314	1407 291 1616	+121.75 ± 5.45	0 46 497 1066 48	+266.1 ± 15.29
ncm-dbt-06	09:57:50	1227384	3358	1435 277 1646	+124.93 ± 5.3	0 49 459 1135 36	+280.58 ± 15.92
			20000	8535 1690 9775	+123.91 ± 2.17	0 257 2892 6600 251	+275.08 ± 6.33

Test Detail

ID	Host	Base NPS	Games	WLD	Standard Elo	Ptnml(0-2)	Gamepair Elo	PGN
195124	ncm-dbt-05	1238640	314	124 27 163	+110.95 ± 16.81	0 3 57 94 3	+240.13 ± 45.23	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1671080417 \ -pgnout ncm-dbt-20230629-0600-042.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195123	ncm-dbt-02	1241196	310	138 24 148	+134.04 ± 16.9	0 1 45 103 6	+299.14 ± 51.23	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2244469358 \ -pgnout ncm-dbt-20230629-0600-041.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195122	ncm-dbt-03	1236175	312	132 23 157	+126.71 ± 15.9	0 1 48 104 3	+287.73 ± 49.42	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2933625931 \ -pgnout ncm-dbt-20230629-0600-040.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195121	ncm-dbt-01	1206628	346	154 32 160	+128.0 ± 15.58	0 3 48 119 3	+293.2 ± 49.89	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2022843234 \ -pgnout ncm-dbt-20230629-0600-039.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195120	ncm-dbt-04	1216210	360	149 24 187	+125.87 ± 14.63	0 3 50 126 1	+293.87 ± 48.84	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2767053758 \ -pgnout ncm-dbt-20230629-0600-038.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195119	ncm-dbt-06	1225906	358	144 42 172	+101.81 ± 15.76	0 7 63 109 0	+224.89 ± 43.3	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3598614322 \ -pgnout ncm-dbt-20230629-0600-037.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195118	ncm-dbt-02	1230338	500	211 46 243	+119.11 ± 13.52	0 6 78 161 5	+263.42 ± 38.85	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 940780920 \ -pgnout ncm-dbt-20230629-0600-036.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195117	ncm-dbt-05	1213079	500	214 51 235	+117.55 ± 14.37	0 7 82 152 9	+249.64 ± 37.86	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2784136160 \ -pgnout ncm-dbt-20230629-0600-035.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195116	ncm-dbt-03	1247106	500	219 35 246	+134.15 ± 12.67	0 2 68 174 6	+309.64 ± 41.56	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 957741493 \ -pgnout ncm-dbt-20230629-0600-034.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195115	ncm-dbt-01	1203968	500	212 45 243	+120.67 ± 13.84	0 6 78 159 7	+263.42 ± 38.85	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1120262785 \ -pgnout ncm-dbt-20230629-0600-033.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195114	ncm-dbt-04	1211826	500	217 47 236	+123.02 ± 13.99	0 8 70 166 6	+273.0 ± 41.13	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 253424151 \ -pgnout ncm-dbt-20230629-0600-032.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195113	ncm-dbt-06	1224339	500	222 43 235	+130.14 ± 13.88	0 6 67 169 8	+290.66 ± 42.08	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3034582320 \ -pgnout ncm-dbt-20230629-0600-031.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195106	ncm-dbt-05	1234492	500	206 38 256	+121.45 ± 13.66	0 8 70 168 4	+273.0 ± 41.13	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2171217807 \ -pgnout ncm-dbt-20230629-0600-030.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195105	ncm-dbt-02	1221541	500	206 39 255	+120.67 ± 12.78	0 4 78 165 3	+273.0 ± 38.74	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 600664570 \ -pgnout ncm-dbt-20230629-0600-029.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195104	ncm-dbt-03	1217373	500	214 41 245	+125.38 ± 14.63	0 10 65 167 8	+275.45 ± 42.59	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2239533975 \ -pgnout ncm-dbt-20230629-0600-028.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195103	ncm-dbt-01	1186178	500	216 46 238	+123.02 ± 13.82	0 8 69 168 5	+275.45 ± 41.43	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 563669290 \ -pgnout ncm-dbt-20230629-0600-027.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195102	ncm-dbt-04	1222681	500	206 47 247	+114.45 ± 15.15	0 10 82 147 11	+236.51 ± 37.9	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1137617483 \ -pgnout ncm-dbt-20230629-0600-026.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195101	ncm-dbt-06	1226369	500	214 41 245	+125.38 ± 13.07	0 5 71 170 4	+285.49 ± 40.81	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2926698034 \ -pgnout ncm-dbt-20230629-0600-025.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195094	ncm-dbt-02	1226116	500	214 38 248	+127.76 ± 14.93	0 8 70 160 12	+273.0 ± 41.13	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1240148445 \ -pgnout ncm-dbt-20230629-0600-024.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195093	ncm-dbt-05	1233998	500	216 46 238	+123.02 ± 14.33	0 8 72 162 8	+268.17 ± 40.54	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 4292768009 \ -pgnout ncm-dbt-20230629-0600-023.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195092	ncm-dbt-03	1241932	500	210 36 254	+126.17 ± 13.95	0 8 66 170 6	+282.94 ± 42.37	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 580797162 \ -pgnout ncm-dbt-20230629-0600-022.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195091	ncm-dbt-04	1254934	500	215 39 246	+127.76 ± 13.02	0 5 68 173 4	+293.29 ± 41.75	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1644342207 \ -pgnout ncm-dbt-20230629-0600-021.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195090	ncm-dbt-01	1208108	500	212 38 250	+126.17 ± 13.06	0 6 67 174 3	+290.66 ± 42.08	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1881173885 \ -pgnout ncm-dbt-20230629-0600-020.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195089	ncm-dbt-06	1219945	500	212 35 253	+128.55 ± 14.26	0 9 62 172 7	+288.06 ± 43.65	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3188717246 \ -pgnout ncm-dbt-20230629-0600-019.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195082	ncm-dbt-03	1241486	500	211 41 248	+123.02 ± 13.82	0 5 78 159 8	+268.17 ± 38.8	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3425490427 \ -pgnout ncm-dbt-20230629-0600-018.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195081	ncm-dbt-02	1216519	500	211 46 243	+119.11 ± 13.86	0 8 74 163 5	+263.42 ± 39.97	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2387010373 \ -pgnout ncm-dbt-20230629-0600-017.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195080	ncm-dbt-05	1228982	500	212 45 243	+120.67 ± 14.01	0 9 70 166 5	+268.17 ± 41.1	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1554745903 \ -pgnout ncm-dbt-20230629-0600-016.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195078	ncm-dbt-01	1182369	500	218 46 236	+124.6 ± 14.8	0 11 64 167 8	+273.0 ± 42.83	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3984591337 \ -pgnout ncm-dbt-20230629-0600-015.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195077	ncm-dbt-04	1207558	500	207 46 247	+116.0 ± 14.85	0 9 81 150 10	+243.0 ± 38.14	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 448301713 \ -pgnout ncm-dbt-20230629-0600-014.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195075	ncm-dbt-06	1224834	500	217 42 241	+126.97 ± 14.45	0 10 62 171 7	+282.94 ± 43.58	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 621317045 \ -pgnout ncm-dbt-20230629-0600-013.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195070	ncm-dbt-03	1250418	500	213 42 245	+123.81 ± 12.35	0 4 72 173 1	+288.06 ± 40.46	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3418343829 \ -pgnout ncm-dbt-20230629-0600-012.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195068	ncm-dbt-05	1236556	500	208 46 246	+116.77 ± 13.54	0 6 81 158 5	+256.44 ± 38.07	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3581907892 \ -pgnout ncm-dbt-20230629-0600-011.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195066	ncm-dbt-02	1218324	500	210 44 246	+119.89 ± 13.33	0 4 82 158 6	+263.42 ± 37.69	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1173512292 \ -pgnout ncm-dbt-20230629-0600-010.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195062	ncm-dbt-04	1201139	500	223 38 239	+134.95 ± 12.84	0 2 68 173 7	+309.64 ± 41.56	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 4259881130 \ -pgnout ncm-dbt-20230629-0600-009.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195061	ncm-dbt-01	1223140	500	209 40 251	+122.24 ± 14.17	0 10 66 169 5	+273.0 ± 42.27	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3754708362 \ -pgnout ncm-dbt-20230629-0600-008.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195059	ncm-dbt-06	1242982	500	212 39 249	+125.38 ± 12.7	0 4 72 171 3	+288.06 ± 40.46	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2053459538 \ -pgnout ncm-dbt-20230629-0600-007.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195054	ncm-dbt-03	1236456	500	228 42 230	+135.76 ± 13.57	0 2 71 166 11	+301.33 ± 40.6	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2805445838 \ -pgnout ncm-dbt-20230629-0600-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195052	ncm-dbt-05	1234081	500	227 38 235	+138.18 ± 14.59	0 5 65 166 14	+301.33 ± 42.75	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 859923718 \ -pgnout ncm-dbt-20230629-0600-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195050	ncm-dbt-02	1241034	500	206 47 247	+114.45 ± 13.38	0 6 83 157 4	+251.89 ± 37.57	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 3638768368 \ -pgnout ncm-dbt-20230629-0600-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195049	ncm-dbt-06	1227315	500	214 35 251	+130.14 ± 14.06	0 8 62 173 7	+293.29 ± 43.72	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1652545399 \ -pgnout ncm-dbt-20230629-0600-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195048	ncm-dbt-04	1235752	500	217 43 240	+126.17 ± 13.77	0 7 68 169 6	+282.94 ± 41.76	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 2539046874 \ -pgnout ncm-dbt-20230629-0600-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14
195047	ncm-dbt-01	1188274	500	215 47 238	+121.45 ± 13.66	0 5 79 159 7	+265.78 ± 38.54	↓
cutechess-cli \ -rounds 259 \ -games 2 \ -concurrency 9 \ -srand 1557886848 \ -pgnout ncm-dbt-20230629-0600-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20230629-0600 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:68e1e9b3811e16cad014b590d7443b9063b3eb52 \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14

Commit

Commit ID	68e1e9b3811e16cad014b590d7443b9063b3eb52
Author	Joost VandeVondele
Date	2023-06-29 06:00:10 UTC
Stockfish 16 Official release version of Stockfish 16 Bench: 2593605 --- Stockfish 16 A new major release of Stockfish is now available at https://stockfishchess.org/download/ Quality of chess play Stockfish continues to demonstrate its ability to discover superior moves with remarkable speed. In self-play against Stockfish 15, this new release gains up to 50 Elo[1] and wins up to 12 times more game pairs[2] than it loses. In major chess engine tournaments, Stockfish reliably tops the rankings[3] winning the TCEC season 24 Superfinal, Swiss, Fischer Random, and Double Random Chess tournaments and the CCC 19 Bullet, 20 Blitz, and 20 Rapid competitions. Leela Chess Zero[4] was the challenger in most finals, putting top-engine chess now firmly in the hands of teams embracing free and open-source software. Progress made This updated version of Stockfish introduces several enhancements, including an upgraded neural net architecture (SFNNv6)[5], improved implementation, and refined parameterization. The ongoing utilization of Leela’s data combined with a novel inference approach exploiting sparsity[6], and network compression[7] ensure a speedy evaluation and modest binary sizes while allowing for more weights and higher accuracy. The search has undergone more optimization, leading to improved performance, particularly in longer analyses[8]. Additionally, the Fishtest framework has been improved and is now able to run the tests needed to validate new ideas with 10000s of CPU cores. Usability improvements Stockfish now comes with documentation, found in the wiki folder when downloading it or on GitHub[9]. Additionally, Stockfish now includes a clear and consistent forced tablebase win score, displaying a value of 200 minus the number of plies required to reach a tablebase win[10]. Furthermore, the UCI_Elo option, to reduce its strength, has been calibrated[11]. It is worth noting that the evaluation system remains consistent with Stockfish 15.1[12], maintaining the choice that 100cp means a 50% chance of winning the game against an equal opponent[13]. Finally, binaries of our latest development version are now provided continuously as pre-releases on GitHub making it easier for enthusiasts to download the latest and strongest version of the program[14], we thank Roman Korba for having provided a similar service for a long time. Thank you The success of the Stockfish project relies on the vibrant community of passionate enthusiasts (we appreciate each and every one of you!) who generously contribute their knowledge, time, and resources. Together, this dedicated community works towards the common goal of developing a powerful, freely accessible, and open-source chess engine. We invite all chess enthusiasts to join the Fishtest testing framework and contribute to the project[15]. The Stockfish team [1] https://tests.stockfishchess.org/tests/view/649409f0dc7002ce609c99cc [2] https://tests.stockfishchess.org/tests/view/649409d7dc7002ce609c99c6 [3] https://en.wikipedia.org/wiki/Stockfish_(chess)#Competition_results [4] https://lczero.org/ [5] https://github.com/official-stockfish/Stockfish/commit/c1fff71 [6] https://github.com/official-stockfish/Stockfish/commit/38e6166 [7] https://github.com/official-stockfish/Stockfish/commit/a46087e [8] https://github.com/official-stockfish/Stockfish/commit/472e726 [9] https://github.com/official-stockfish/Stockfish/wiki/ [10] https://github.com/official-stockfish/Stockfish/commit/def2966 [11] https://github.com/official-stockfish/Stockfish/commit/a08b8d4 [12] https://github.com/official-stockfish/Stockfish/commit/52e84e4 [13] https://github.com/official-stockfish/Stockfish/wiki/Stockfish-FAQ#interpretation-of-the-stockfish-evaluation [14] https://github.com/official-stockfish/Stockfish/releases?q=prerelease%3Atrue [15] https://stockfishchess.org/get-involved/