NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.
| Host | Duration | Avg Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo |
|---|
| ID | Host | Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo | CLI | PGN |
|---|
| Commit ID | 4b0043ae7c92507d690eb4c4eb6249550d0891ff |
|---|---|
| Author | Joost Vandevondele |
| Date | 2016-09-18 06:13:34 UTC |
|
Use fixed depth bench to make PGO builds more reproducible
Discussed on fishcooking
proposal and objdump verification:
https://groups.google.com/d/msg/fishcooking/4_ausUwMXP0/EGPsMYqOFAAJ
verified no significant speed difference between depth and time:
https://groups.google.com/d/msg/fishcooking/4_ausUwMXP0/KazW5QZmFgAJ
stockfish_time - stats:
mean = 2207232.56 std = 7079.51 std/mean = 0.003207
stockfish_depth - stats:
mean = 2201783.57 std = 6356.69 std/mean = 0.002887
No functional change
|
|