LJ38 double-ended saddle

SystemLennard-Jones

Atoms38

Runs50

Cap2500 force calls

The main metric is Force calls. Scoring rules.

Charts

Success rate on LJ38 double-ended saddle, in rank order. The rate is the share of runs that finished. 8 ranked rows did not record a success rate and are not drawn. PDF.

Dolan-Moré profile for Minimum mode on LJ38 double-ended saddle. The curve is the fraction of starts whose cost is at most tau times the best finite cost in this family. Tau is 1, 2, 4, 8, and 16. An unsolved start stays in the denominator and does not cross a finite tau. Dolan, E. D. and Moré, J. J., Math. Program. 91:201-213, 2002. PDF.

Force calls on each recorded start for LJ38 double-ended saddle. The box is the quartiles of those counts. A blue point is a success and a vermillion cross is a recorded failure (402). The horizontal position is the recorded count. The vertical offset only keeps marks from covering one another. PDF.

eOn on LJ38 double-ended saddle. Each point is one recorded version of one method. Versions run left to right in numeric order. PAR-2 is omitted where that row has none. No line is drawn across a missing value. PDF.

Performance profile. Higher means more starts solved within that cost ratio. Colored lines are the six largest areas. Thin lines are the other 23.

• Eon svn (r2549) · Lanczos improved L-BFGS
• eOn (3.5.0) · Lanczos improved L-BFGS
• Eon svn (r2549) · Lanczos L-BFGS
• eOn (3.5.0) · Lanczos L-BFGS
• Eon svn (r2549) · Dimer L-BFGS
• eOn (3.5.0) · CI-NEB

Cost distribution of 35 ranked rows. Logarithmic bins, from 131 to 5000.

Performance profile SVG Cost and success SVG

Performance profile. Select the plot for the full SVG.

Cost and success. Select the plot for the full SVG.

A shorter bar is a lower cost on a log scale shared by the codes of this problem.

Leaderboard

A shorter bar is a lower cost on the log scale for that problem. Select a column heading to sort.

Rank	Code	Best	Median	Success	PAR-2
1	OPTIM	131	131	100.0%	131
2	Eon svn	235.2	785.24	100.0%	235.2
3	eOn	259.46	1010.08	100.0%	259.46
4	pele	293	293	100.0%	293
5	geodesic	1208.42	2452.66	86.0%	1208.42
6	COMPASS	1517.3	1517.3	94.0%	1517.3
7	Sella	2055.3	2055.3	100.0%	2055.3
8	rgsaddle	2145.72	2331.95	78.0%	2145.72
9	TSASE	3119.56	4059.78	52.0%	3119.56
10	LAMMPS	4526.2	4737.95	16.0%	4526.2
11	ASE	4912.84	4912.84	2.0%	4912.84

The tables below list every ranked row, by method family and overall.

eOn band rows are not yet on the harness counter.

A run short of the full set, or a run that is not the stock method, stays off the ranked table.

Files with no per-start harness record stay off the ranked table.

Band / NEB

Band / NEB. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	eOn (3.5.0) · CI-NEB	1003.88	force calls	0	0	50
2	Eon svn (r2549) · CI-NEB	1100.84	force calls	0	0	50
3	geodesic (1.0.0) · CI-NEB-LBFGS geodesic	1208.42	force calls	0	0	50
4	eOn ((develop 500350d9) · Doubly nudged CI-NEB) · Doubly nudged CI-NEB	1228.96	force calls	0	0	50
5	eOn (r2156) · eon-ci-swdneb	1274.9	force calls	0	0	50
6	Eon svn (r2549) · Doubly nudged CI-NEB	1294.86	force calls	0	0	50
7	eOn (r2156) · eon-ci-neb	1411.22	force calls	0	0	50
8	Sella (2.6.0) · Sella+NEB	2055.3	force calls	0	0	50
9	rgsaddle (0.2.2) · CI-NEB energy-weighted	2145.72	force calls	0	0	50
10	rgsaddle (0.2.2) · CI-NEB	2272.18	force calls	0	0	50
11	rgsaddle (0.2.2) · OCI-NEB	2289.86	force calls	0	0	50
12	rgsaddle (0.2.2) · IDPP CI-NEB	2374.04	force calls	0	0	50
13	geodesic (1.0.0) · CI-NEB geodesic	2452.66	force calls	0	0	50
14	rgsaddle (0.2.2) · S-IDPP CI-NEB	2473.46	force calls	0	0	50
15	TSASE (1.0) · CI-NEB	3119.56	force calls	0	0	50
16	LAMMPS (30 Sep 2026) · CI-NEB-fire	4526.2	force calls	0	0	50
17	geodesic (1.0.0) · NEB-FIRE geodesic	4889.98	force calls	0	0	50
18	ASE (3.29.0) · NEB-FIRE	4912.84	force calls	0	0	50
19	LAMMPS (30 Sep 2026) · CI-NEB-quickmin	4949.7	force calls	0	0	50
20	TSASE (1.0) · NEB	5000	force calls	0	0	50
21	rgsaddle (0.2.2) · NEB	5000	force calls	0	0	50

Minimum mode

Minimum mode. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	Eon svn (r2549) · Lanczos improved L-BFGS	235.2	force calls	0	0	50
2	eOn (3.5.0) · Lanczos improved L-BFGS	259.46	force calls	0	0	50
3	eOn (r2156) · eon-lanczos-improved	287	force calls	0	0	50
4	Eon svn (r2549) · Lanczos L-BFGS	299.82	force calls	0	0	50
5	eOn (r2156) · eon-dimer	337	force calls	0	0	50
6	eOn (3.5.0) · Lanczos L-BFGS	342.28	force calls	0	0	50
7	Eon svn (r2549) · Dimer L-BFGS	469.64	force calls	0	0	50
8	eOn (3.5.0) · Dimer L-BFGS	2025.86	force calls	0	0	50

Connect / double-ended

Connect / double-ended. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	OPTIM · DNEB + hybrid eigenvector following	131	force calls	0	0	50
2	pele · hybrid eigenvector following	293	force calls	0	0	50
3	eOn (3.6.0) · LBFGS	1010.08	force calls	0	0	50
4	eOn (develop f6817b53) (3.6.0) · LBFGS	1077.94	force calls	0	0	50
5	Eon svn (r2549) · LBFGS	1180.1	force calls	0	0	50
6	COMPASS (1.0.0) · COMPASS	1517.3	force calls	0	0	50

Overall

One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate. The per-image mean divides the total by the number of moving images.

Row	Family	Family rank	Unit	PAR-2	Published	Median calls	Mean calls	Per-image mean	Hessians	FD calls	Failures	Runs
OPTIM · DNEB + hybrid eigenvector following	Connect / double-ended	1	force calls	131	131	120	131		0	0	0 failed	50
Eon svn (r2549) · Lanczos improved L-BFGS	Minimum mode	1	force calls	235.2		197	235.2	47.04	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	50
eOn (3.5.0) · Lanczos improved L-BFGS	Minimum mode	2	force calls	259.46		212	259.46	51.892	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	50
eOn (r2156) · eon-lanczos-improved	Minimum mode	3	force calls	287	287	201	287		0	0	0 failed	50
pele · hybrid eigenvector following	Connect / double-ended	2	force calls	293	293	222	293		0	0	0 failed	50
Eon svn (r2549) · Lanczos L-BFGS	Minimum mode	4	force calls	299.82		243.5	299.82	59.964	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	50
eOn (r2156) · eon-dimer	Minimum mode	5	force calls	337	337	203.5	337		0	0	0 failed	50
eOn (3.5.0) · Lanczos L-BFGS	Minimum mode	6	force calls	342.28		258.5	342.28	68.456	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	50
Eon svn (r2549) · Dimer L-BFGS	Minimum mode	7	force calls	469.64		206.5	419.64	83.928	0	0	crash 0, divergence 0, cap 1, wrong_stationary 0	50
eOn (3.5.0) · CI-NEB	Band / NEB	1	force calls	1003.88		692	853.88	170.776	0	0	crash 0, divergence 0, cap 3, wrong_stationary 0	50
eOn (3.6.0) · LBFGS	Connect / double-ended	3	force calls	1010.08		692	860.08	172.016	0	0	crash 0, divergence 0, cap 3, wrong_stationary 0	50
eOn (develop f6817b53) (3.6.0) · LBFGS	Connect / double-ended	4	force calls	1077.94		692	877.94	175.588	0	0	crash 0, divergence 0, cap 4, wrong_stationary 0	50
Eon svn (r2549) · CI-NEB	Band / NEB	2	force calls	1100.84		662	900.84	180.168	0	0	crash 0, divergence 0, cap 4, wrong_stationary 0	50
Eon svn (r2549) · LBFGS	Connect / double-ended	5	force calls	1180.1		657	930.1	186.02	0	0	crash 0, divergence 0, cap 5, wrong_stationary 0	50
geodesic (1.0.0) · CI-NEB-LBFGS geodesic	Band / NEB	3	force calls	1208.42		512	768.74	153.748	0	0	crash 0, divergence 3, cap 4, wrong_stationary 0	50
eOn ((develop 500350d9) · Doubly nudged CI-NEB) · Doubly nudged CI-NEB	Band / NEB	4	force calls	1228.96		664.5	928.96	185.792	0	0	crash 0, divergence 0, cap 6, wrong_stationary 0	50
eOn (r2156) · eon-ci-swdneb	Band / NEB	5	force calls	1274.9	861	793.5	861		0	0	5 failed, class not recorded	50
Eon svn (r2549) · Doubly nudged CI-NEB	Band / NEB	6	force calls	1294.86		662	994.86	198.972	0	0	crash 0, divergence 0, cap 6, wrong_stationary 0	50
eOn (r2156) · eon-ci-neb	Band / NEB	7	force calls	1411.22	827	788.5	827		0	0	7 failed, class not recorded	50
COMPASS (1.0.0) · COMPASS	Connect / double-ended	6	force calls	1517.3		1234	1367.3	273.46	0	0	crash 0, divergence 0, cap 3, wrong_stationary 0	50
eOn (3.5.0) · Dimer L-BFGS	Minimum mode	8	force calls	2025.86		414.5	1174.46	234.892	0	0	crash 0, divergence 1, cap 16, wrong_stationary 0	50
Sella (2.6.0) · Sella+NEB	Band / NEB	8	force calls	2055.3		2047.5	2055.3	411.06	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	50
rgsaddle (0.2.2) · CI-NEB energy-weighted	Band / NEB	9	force calls	2145.72		1461.5	1595.72	319.144	0	0	crash 0, divergence 0, cap 11, wrong_stationary 0	50
rgsaddle (0.2.2) · CI-NEB	Band / NEB	10	force calls	2272.18		1484.5	1617.82	323.564	0	0	crash 0, divergence 1, cap 12, wrong_stationary 0	50
rgsaddle (0.2.2) · OCI-NEB	Band / NEB	11	force calls	2289.86		1508.5	1639.86	327.972	0	0	crash 0, divergence 0, cap 13, wrong_stationary 0	50
rgsaddle (0.2.2) · IDPP CI-NEB	Band / NEB	12	force calls	2374.04		1614.5	1670.18	334.036	0	0	crash 0, divergence 1, cap 13, wrong_stationary 0	50
geodesic (1.0.0) · CI-NEB geodesic	Band / NEB	13	force calls	2452.66		1629.5	1643.74	328.748	0	0	crash 0, divergence 2, cap 14, wrong_stationary 0	50
rgsaddle (0.2.2) · S-IDPP CI-NEB	Band / NEB	14	force calls	2473.46		1454.5	1664.2	332.84	0	0	crash 0, divergence 1, cap 15, wrong_stationary 0	50
TSASE (1.0) · CI-NEB	Band / NEB	15	force calls	3119.56		1833	1726.98	345.396	0	0	crash 0, divergence 7, cap 17, wrong_stationary 0	50
LAMMPS (30 Sep 2026) · CI-NEB-fire	Band / NEB	16	force calls	4526.2		2065	2055.2	411.04	0	0	crash 0, divergence 40, cap 0, wrong_stationary 2	50
geodesic (1.0.0) · NEB-FIRE geodesic	Band / NEB	17	force calls	4889.98		2500	2489.98	497.996	0	0	crash 0, divergence 0, cap 48, wrong_stationary 0	50
ASE (3.29.0) · NEB-FIRE	Band / NEB	18	force calls	4912.84		2501	2463.82	492.764	0	0	crash 0, divergence 0, cap 49, wrong_stationary 0	50
LAMMPS (30 Sep 2026) · CI-NEB-quickmin	Band / NEB	19	force calls	4949.7		2485	2485	497	0	0	crash 0, divergence 48, cap 0, wrong_stationary 1	50
TSASE (1.0) · NEB	Band / NEB	20	force calls	5000		1633	1652.86	330.572	0	0	crash 0, divergence 36, cap 14, wrong_stationary 0	50
rgsaddle (0.2.2) · NEB	Band / NEB	21	force calls	5000		2478.5	2126.6	425.32	0	0	crash 0, divergence 25, cap 25, wrong_stationary 0	50

Run under a different budget

These archived rows record a run above this problem's cap and no failure for it, so they ran under a larger cap. This table keeps them apart from the ranking above: eOn (r2156) · eon-lanczos (its archived maximum, 2794, is above the cap of 2500); OPTIM · climbing string (its archived maximum, 2792, is above the cap of 2500).

One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Row	Family	Family rank	Unit	PAR-2	Published	Median calls	Mean calls	Hessians	FD calls	Failures	Runs
eOn (r2156) · eon-lanczos	Minimum mode		force calls		364	241	364	0	0	0 failed	50
OPTIM · climbing string	Connect / double-ended		force calls		786	625	786	0	0	3 failed, class not recorded	50

Off the ranked table

• eOn (3.5.0) · Doubly nudged CI-NEB n=50, not yet on the harness counter.
• eOn (3.6.0) · LBFGS n=50, not yet on the harness counter.
• eOn (3.6.0) · LBFGS n=50, not yet on the harness counter.

Not scored

• Eon: no per-start record
• Eon: no per-start record