Dev Builds » 20120916-0657

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host	Duration	Avg Base NPS	Games	Wins	Losses	Draws	Elo
ncm-et-3	08:51:04	1976880	3671	73	2519	1079	-279.36 ± 10.36
ncm-et-4	08:51:56	1979807	3664	70	2417	1177	-263.74 ± 9.89
ncm-et-9	08:51:16	1976136	3683	78	2501	1104	-274.15 ± 10.24
ncm-et-10	08:50:39	1960727	3659	72	2479	1108	-274.12 ± 10.21
ncm-et-13	04:01:24	1927120	1633	39	1126	468	-278.95 ± 15.76
ncm-et-15	08:50:22	1976483	3690	74	2534	1082	-279.59 ± 10.34
			20000	406	13576	6018	-274.53 ± 4.38

Test Detail

ID	Host	Started (UTC)	Duration	Base NPS	Games	Wins	Losses	Draws	Elo	PGN
117353	ncm-et-9	2020-02-26 02:54	00:00:40	1961693	2	0	1	1	-190.5 ± 634.21	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3531968304 \ -pgnout /tmp/ncm-dbt-20120916-0657-045.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.47+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117352	ncm-et-4	2020-02-26 01:42	01:12:40	1968813	498	5	314	179	-252.16 ± 25.18	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 524573253 \ -pgnout /tmp/ncm-dbt-20120916-0657-044.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.38+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117351	ncm-et-3	2020-02-26 01:42	01:12:25	1962610	500	14	348	138	-280.42 ± 29.14	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 252754356 \ -pgnout /tmp/ncm-dbt-20120916-0657-043.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.46+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117350	ncm-et-9	2020-02-26 01:41	01:12:14	1959240	500	14	334	152	-263.42 ± 27.74	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1621238180 \ -pgnout /tmp/ncm-dbt-20120916-0657-042.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.50+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117349	ncm-et-15	2020-02-26 01:41	01:12:47	1973589	500	4	348	148	-293.29 ± 27.92	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 985783141 \ -pgnout /tmp/ncm-dbt-20120916-0657-041.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.32+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117348	ncm-et-10	2020-02-26 01:41	01:12:56	1964154	500	8	326	166	-261.07 ± 26.36	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 242398336 \ -pgnout /tmp/ncm-dbt-20120916-0657-040.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.44+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117347	ncm-et-4	2020-02-26 00:29	01:12:08	1971564	500	17	328	155	-253.02 ± 27.47	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2363509184 \ -pgnout /tmp/ncm-dbt-20120916-0657-039.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.35+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117346	ncm-et-9	2020-02-26 00:29	01:11:35	1969118	500	10	336	154	-270.57 ± 27.5	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1628277222 \ -pgnout /tmp/ncm-dbt-20120916-0657-038.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.38+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117345	ncm-et-10	2020-02-26 00:28	01:11:40	1967704	500	8	350	142	-290.66 ± 28.66	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2080128961 \ -pgnout /tmp/ncm-dbt-20120916-0657-037.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.39+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117344	ncm-et-15	2020-02-26 00:27	01:12:30	1970331	500	11	345	144	-280.42 ± 28.5	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1175122096 \ -pgnout /tmp/ncm-dbt-20120916-0657-036.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.36+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117343	ncm-et-3	2020-02-26 00:27	01:13:24	1958012	500	11	345	144	-280.42 ± 28.5	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2080191661 \ -pgnout /tmp/ncm-dbt-20120916-0657-035.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.51+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117342	ncm-et-10	2020-02-25 23:16	01:11:12	1967545	500	13	329	158	-258.75 ± 27.17	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1742473850 \ -pgnout /tmp/ncm-dbt-20120916-0657-034.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.40+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117341	ncm-et-9	2020-02-25 23:15	01:12:28	1968637	500	16	337	147	-264.6 ± 28.22	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3926809820 \ -pgnout /tmp/ncm-dbt-20120916-0657-033.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.38+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117340	ncm-et-4	2020-02-25 23:15	01:13:13	1967879	500	15	343	142	-273.0 ± 28.72	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1321864938 \ -pgnout /tmp/ncm-dbt-20120916-0657-032.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.39+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117339	ncm-et-3	2020-02-25 23:15	01:11:19	1962917	500	5	345	150	-288.06 ± 27.75	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3511080538 \ -pgnout /tmp/ncm-dbt-20120916-0657-031.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.45+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117338	ncm-et-15	2020-02-25 23:14	01:12:09	1973739	500	3	345	152	-290.66 ± 27.47	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 178511191 \ -pgnout /tmp/ncm-dbt-20120916-0657-030.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.32+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117337	ncm-et-9	2020-02-25 22:02	01:12:28	1968622	500	10	363	127	-305.45 ± 30.4	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3901328925 \ -pgnout /tmp/ncm-dbt-20120916-0657-029.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.38+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117336	ncm-et-4	2020-02-25 22:02	01:12:26	1967540	500	12	344	144	-277.93 ± 28.51	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1557009377 \ -pgnout /tmp/ncm-dbt-20120916-0657-028.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.40+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117335	ncm-et-3	2020-02-25 22:02	01:12:12	1965842	500	10	341	149	-276.69 ± 27.98	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1266982885 \ -pgnout /tmp/ncm-dbt-20120916-0657-027.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.42+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117334	ncm-et-10	2020-02-25 22:02	01:13:04	1964460	500	6	330	164	-268.17 ± 26.47	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3966598350 \ -pgnout /tmp/ncm-dbt-20120916-0657-026.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.43+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
117333	ncm-et-15	2020-02-25 22:01	01:11:26	1964917	500	11	338	151	-271.78 ± 27.8	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 688882447 \ -pgnout /tmp/ncm-dbt-20120916-0657-025.pgn \ -openings \ file=/home/chess/www/releases/20200221130520/engines/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.43+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/engines/e5_2670/devbuild/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/engines/e5_2670/stockfish/7/stockfish
44923	ncm-et-13	2018-07-08 05:16	00:19:49	1917625	133	3	99	31	-316.65 ± 62.84	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2772179499 \ -pgnout /tmp/ncm-dbt-20120916-0657-024.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=25.03+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44922	ncm-et-10	2018-07-08 05:13	00:23:19	1982323	159	1	104	54	-268.05 ± 46.27	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 4165083191 \ -pgnout /tmp/ncm-dbt-20120916-0657-023.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.21+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44921	ncm-et-3	2018-07-08 05:12	00:24:13	1992088	171	2	117	52	-283.27 ± 47.62	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2669892003 \ -pgnout /tmp/ncm-dbt-20120916-0657-022.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.10+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44920	ncm-et-4	2018-07-08 05:11	00:25:09	1991456	166	2	102	62	-242.14 ± 43.05	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3862047149 \ -pgnout /tmp/ncm-dbt-20120916-0657-021.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.10+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44919	ncm-et-9	2018-07-08 05:10	00:25:45	1990190	181	1	129	51	-306.27 ± 48.04	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3954295757 \ -pgnout /tmp/ncm-dbt-20120916-0657-020.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.12+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44918	ncm-et-15	2018-07-08 05:10	00:26:17	1983890	190	6	131	53	-274.16 ± 47.43	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 694450316 \ -pgnout /tmp/ncm-dbt-20120916-0657-019.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.19+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44917	ncm-et-13	2018-07-08 04:02	01:13:37	1949754	500	11	339	150	-273.0 ± 27.9	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3020977523 \ -pgnout /tmp/ncm-dbt-20120916-0657-018.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.62+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44916	ncm-et-10	2018-07-08 04:01	01:10:37	1981228	500	8	349	143	-289.36 ± 28.56	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3782640667 \ -pgnout /tmp/ncm-dbt-20120916-0657-017.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.23+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44915	ncm-et-9	2018-07-08 03:59	01:10:23	1989401	500	9	343	148	-280.42 ± 28.06	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2920519394 \ -pgnout /tmp/ncm-dbt-20120916-0657-016.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.13+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44914	ncm-et-15	2018-07-08 03:58	01:10:50	1981541	500	11	350	139	-286.77 ± 29.03	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3245529449 \ -pgnout /tmp/ncm-dbt-20120916-0657-015.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.22+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44913	ncm-et-4	2018-07-08 03:58	01:12:20	1990981	500	7	329	164	-265.78 ± 26.51	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3091170681 \ -pgnout /tmp/ncm-dbt-20120916-0657-014.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.11+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44912	ncm-et-3	2018-07-08 03:58	01:13:18	1992722	500	14	339	147	-269.37 ± 28.22	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1015014967 \ -pgnout /tmp/ncm-dbt-20120916-0657-013.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.09+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44911	ncm-et-9	2018-07-08 02:46	01:11:49	1988454	500	8	332	160	-268.17 ± 26.89	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3038111119 \ -pgnout /tmp/ncm-dbt-20120916-0657-012.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.14+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44910	ncm-et-10	2018-07-08 02:46	01:14:19	1956559	500	11	357	132	-295.94 ± 29.81	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 3024380659 \ -pgnout /tmp/ncm-dbt-20120916-0657-011.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.53+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44909	ncm-et-4	2018-07-08 02:46	01:11:02	1989717	500	7	336	157	-274.22 ± 27.14	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1939329261 \ -pgnout /tmp/ncm-dbt-20120916-0657-010.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.12+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44908	ncm-et-15	2018-07-08 02:45	01:11:44	1981382	500	10	349	141	-286.77 ± 28.8	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2874451356 \ -pgnout /tmp/ncm-dbt-20120916-0657-009.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.23+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44907	ncm-et-3	2018-07-08 02:45	01:11:43	1990822	500	8	350	142	-290.66 ± 28.66	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 4215282040 \ -pgnout /tmp/ncm-dbt-20120916-0657-008.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.11+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44906	ncm-et-13	2018-07-08 02:43	01:17:26	1850277	500	11	348	141	-284.22 ± 28.81	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1130042024 \ -pgnout /tmp/ncm-dbt-20120916-0657-007.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=25.94+0.26 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44905	ncm-et-4	2018-07-08 01:31	01:12:58	1990507	500	5	321	174	-258.75 ± 25.59	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 8290318 \ -pgnout /tmp/ncm-dbt-20120916-0657-006.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.11+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44904	ncm-et-13	2018-07-08 01:31	01:10:32	1990825	500	14	340	146	-270.57 ± 28.32	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1587678225 \ -pgnout /tmp/ncm-dbt-20120916-0657-005.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.11+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44903	ncm-et-3	2018-07-08 01:31	01:12:30	1990032	500	9	334	157	-269.37 ± 27.19	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 4168440818 \ -pgnout /tmp/ncm-dbt-20120916-0657-004.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.12+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44902	ncm-et-15	2018-07-08 01:31	01:12:39	1982479	500	18	328	154	-251.89 ± 27.56	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 1706886664 \ -pgnout /tmp/ncm-dbt-20120916-0657-003.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.21+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44901	ncm-et-9	2018-07-08 01:31	01:13:54	1989874	500	10	326	164	-258.75 ± 26.59	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 437912522 \ -pgnout /tmp/ncm-dbt-20120916-0657-002.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=24.12+0.24 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish
44900	ncm-et-10	2018-07-08 01:31	01:13:32	1901843	500	17	334	149	-259.91 ± 28.03	↓
/home/chess/cutechess/projects/cli/cutechess-cli \ -games 500 \ -concurrency 8 \ -srand 2264746486 \ -pgnout /tmp/ncm-dbt-20120916-0657-001.pgn \ -openings \ file=/home/chess/www/releases/20180707211544/stockfish/books/8moves_v3.pgn \ format=pgn \ order=random \ plies=16 \ -repeat \ -resign \ movecount=3 \ score=400 \ -draw \ movenumber=34 \ movecount=8 \ score=20 \ -each \ tc=25.24+0.25 \ timemargin=10000 \ proto=uci \ option.Hash=128 \ option.Threads=2 \ -engine \ name=sf-20120916-0657 \ cmd=/home/chess/devbuilds/bin/x86-64-modern/20120916-0657-4b7dbb3922a0c563d74be4474b0cbdc1b00b4595/stockfish \ -engine \ name=sf7 \ cmd=/home/chess/stockfish/7/bin/stockfish

Commit

Commit ID	4b7dbb3922a0c563d74be4474b0cbdc1b00b4595
Author	Marco Costalba
Date	2012-09-16 06:57:26 UTC
Fix KpsK endgame Broken by commit a44c5cf4f77b05a03 of 3 /12 / 2011 that was labeled "No functional change" because our 'bench' test didn't triggered that particular endgame. Indeed we need to run a specific bench on a set of endgames position when touching endgame.cpp because normal bench does not cover endgames properly. Found by MSVC 2012 code analyzer.