Pt heptamer island

SystemMorse

Atoms343

Runs—¹

Cap—²

¹ The problem file does not state a run count, and the ranked rows do not agree. They record 2 on eOn; 16 on Sella; 5000 on softsaddle; 20000 on rgsaddle.

² The problem file does not state a cap.

A 2-search SoftSaddle batch stays off this rank. The April 2019 adaptive_HSphere-lbfgs-davidson-sm50 entry records 423491.1 force calls.

The main metric is Force calls. Scoring rules.

Charts

Success rate on Pt heptamer island, in rank order. The rate is the share of runs that finished. 12 ranked rows did not record a success rate and are not drawn. PDF.

Force calls on each recorded start for Pt heptamer island. The box is the quartiles of those counts. A blue point is a success and a vermillion cross is a recorded failure (89998). The horizontal position is the recorded count. The vertical offset only keeps marks from covering one another. PDF.

Eon on Pt heptamer island. Each point is one recorded version of one method. Versions run left to right in numeric order. PAR-2 is omitted where that row has none. No line is drawn across a missing value. PDF.

Cost distribution of 8 ranked rows. Logarithmic bins, from 5201.62 to 2e+08.

No performance profile. The ranked rows do not share one start set.

Cost and success SVG

Cost and success for Pt heptamer island — Cost and success. Select the plot for the full SVG.

A shorter bar is a lower cost on a log scale shared by the codes of this problem.

Leaderboard

A shorter bar is a lower cost on the log scale for that problem. Select a column heading to sort.
Rank	Code	Best	Median	Success	PAR-2
1	Sella	5201.62	5201.62	75.0%	5201.62
2	SoftSaddle	10124.2	10126.1	0.0%	10124.2
3	rgsaddle	10128	10128	0.0%	10128
4	eOn	2e+08	2e+08	0.0%	2e+08

The tables below list every ranked row, by method family and overall.

Eon rows are not yet on the harness counter.

A run short of the full set, or a run that is not the stock method, stays off the ranked table.

These archived rows rank on the value each one published, which leaves out any failure. The failure rule has no score for them, for the reason given with each.

SoftSaddle · adaptive_HSphere-lbfgs-davidson-sm50 (the entry records no failure count)
SoftSaddle · adaptive_HSphere-lbfgs-lanczos-sm50 (the entry records no failure count)
SoftSaddle · adaptive_HSphere-lbfgs-lanczos-sm0 (the entry records no failure count)
eOn (r2213) · eon-dimer-lbfgs-5-rotations-20-bowl_breakout (the entry records no failure count)
eOn (r2205) · eon-md (the entry records no failure count)
eOn (r2213) · eon-dimer-lbfgs-5-rotations (the entry records no failure count)
eOn (r2213) · eon-dimer-lbfgs-20-rotations-20-bowl_breakout (the entry records no failure count)
eOn (r2213) · eon-dimer-lbfgs-20-rotations (the entry records no failure count)
eOn (r2213) · eon-dimer-cg-5-rotations-20-bowl_breakout (the entry records no failure count)
eOn (r2205) · eon-dimer (the entry records no failure count)
eOn (r2213) · eon-dimer-cg-20-rotations-20-bowl_breakout (the entry records no failure count)
eOn (r2213) · eon-dimer-cg-20-rotations (the entry records no failure count)

Minimum mode

Minimum mode. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.
Rank	Row	Cost	Unit	Runs
1	SoftSaddle (2019.04) · adaptive hypersphere LBFGS Lanczos	10124.2	force calls per search	5000
2	rgsaddle (0.2.2) · Dimer FIRE	10128	force calls per search	20000
3	rgsaddle (0.2.2) · Dimer L-BFGS	10128	force calls per search	20000
4	rgsaddle (0.2.2) · Dimer rotation-skip L-BFGS	10128	force calls per search	20000
5	rgsaddle (0.2.2) · Lanczos L-BFGS	10128	force calls per search	20000
6	eOn (3.6.0) · Lanczos	2e+08	force calls per search	2

Single-ended saddle

Single-ended saddle. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.
Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	Sella (2.6.0) · Sella	5201.62	force calls per search	0	0	16
2	SoftSaddle (2019.04) · adaptive hypersphere LBFGS Davidson	10128	force calls per search	0	0	5000

Overall

One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.
Row	Family	Family rank	Unit	PAR-2	Median calls	Mean calls	Failures	Runs
Sella (2.6.0) · Sella	Single-ended saddle	1	force calls per search	5201.62	225	247.812	crash 0, divergence 0, cap 0, wrong_stationary 4	16
SoftSaddle (2019.04) · adaptive hypersphere LBFGS Lanczos	Minimum mode	1	force calls per search	10124.2	389	431.839	crash 0, divergence 4884, cap 108, wrong_stationary 6	5000
SoftSaddle (2019.04) · adaptive hypersphere LBFGS Davidson	Single-ended saddle	2	force calls per search	10128	1060	1059.76	crash 0, divergence 0, cap 5000, wrong_stationary 0	5000
rgsaddle (0.2.2) · Dimer FIRE	Minimum mode	2	force calls per search	10128	1013	1011.03	crash 0, divergence 1, cap 19999, wrong_stationary 0	20000
rgsaddle (0.2.2) · Dimer L-BFGS	Minimum mode	3	force calls per search	10128	1013	1012.16	crash 0, divergence 0, cap 20000, wrong_stationary 0	20000
rgsaddle (0.2.2) · Dimer rotation-skip L-BFGS	Minimum mode	4	force calls per search	10128	441	479.98	crash 0, divergence 19718, cap 262, wrong_stationary 20	20000
rgsaddle (0.2.2) · Lanczos L-BFGS	Minimum mode	5	force calls per search	10128	1002	998.34	crash 0, divergence 4068, cap 15932, wrong_stationary 0	20000
eOn (3.6.0) · Lanczos	Minimum mode	6	force calls per search	2e+08	3.98507e+06	3.98507e+06	crash 0, divergence 0, cap 2, wrong_stationary 0	2

A row that counts another unit is ranked in its own table after the overall table.

Ranked in force calls per campaign

These rows count force calls per campaign. The tables above count force calls per search, so these rows are ranked only among themselves.

The cost in the v2 scoring table is one displacement search. The archived force_calls is the mean over campaigns of each campaign's total: done.py and get_data_for_web_benchmark.m add every search until the campaign ends, and the benchmark website's Nf is the total number of force calls. force_calls / total_searches is the mean cost of one search, but the archive states no per-search cap. The PAR-2 that ranks the harness rows needs one, so the archived rows keep their own unit.

One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.
Row	Family	Family rank	Unit	Published	Mean calls	Failures	Runs
SoftSaddle · adaptive_HSphere-lbfgs-davidson-sm50	Single-ended saddle	1	force calls per campaign	423491	423491	not recorded	10
SoftSaddle · adaptive_HSphere-lbfgs-lanczos-sm50	Minimum mode	1	force calls per campaign	814519	814519	not recorded	10
SoftSaddle · adaptive_HSphere-lbfgs-lanczos-sm0	Minimum mode	2	force calls per campaign	1.00343e+06	1.00343e+06	not recorded	10
eOn (r2213) · eon-dimer-lbfgs-5-rotations-20-bowl_breakout	Minimum mode	3	force calls per campaign	1.34547e+06	1.34547e+06	not recorded	9
eOn (r2205) · eon-md	Single-ended saddle	2	force calls per campaign	1.47817e+06	1.47817e+06	not recorded	9
eOn (r2213) · eon-dimer-lbfgs-5-rotations	Minimum mode	4	force calls per campaign	1.68444e+06	1.68444e+06	not recorded	9
eOn (r2213) · eon-dimer-lbfgs-20-rotations-20-bowl_breakout	Minimum mode	5	force calls per campaign	2.73231e+06	2.73231e+06	not recorded	9
eOn (r2213) · eon-dimer-lbfgs-20-rotations	Minimum mode	6	force calls per campaign	3.34149e+06	3.34149e+06	not recorded	9
eOn (r2213) · eon-dimer-cg-5-rotations-20-bowl_breakout	Minimum mode	7	force calls per campaign	3.4637e+06	3.4637e+06	not recorded	9
eOn (r2205) · eon-dimer	Minimum mode	8	force calls per campaign	3.77888e+06	3.77888e+06	not recorded	9
eOn (r2213) · eon-dimer-cg-20-rotations-20-bowl_breakout	Minimum mode	9	force calls per campaign	7.17421e+06	7.17421e+06	not recorded	9
eOn (r2213) · eon-dimer-cg-20-rotations	Minimum mode	10	force calls per campaign	7.94707e+06	7.94707e+06	not recorded	9

Off the ranked table

eOn (3.5.0) n=10, not yet on the harness counter.
eOn (3.5.0) n=10, not yet on the harness counter.

Success on this problem: index 1 at eigenvalue tolerance 1e-6, force norm at the stop, barrier below 1.5 eV, and exactly one minimized endpoint the same structure as the minimized minimum.

Two structures are the same when every atom is closer than distance_difference to its own position in the other (minimum image, atoms by index), as eOn Matter::compare.

A connected saddle within energy_difference of the barrier of a connected saddle already found, and the same structure as it, is a repeat (eOn AKMCState.find_repeat).

Only connected saddles enter that table.