Lennard-Jones 38

SystemLennard-Jones

Atoms38

Runs200

Cap10000 force calls

The main metric is Force calls. Scoring rules.

Charts

Success rate on Lennard-Jones 38, in rank order. The rate is the share of runs that finished. 9 ranked rows did not record a success rate and are not drawn. PDF.

Dolan-Moré profile for Minimum mode on Lennard-Jones 38. The curve is the fraction of starts whose cost is at most tau times the best finite cost in this family. Tau is 1, 2, 4, 8, and 16. An unsolved start stays in the denominator and does not cross a finite tau. Dolan, E. D. and Moré, J. J., Math. Program. 91:201-213, 2002. Colored lines are the six largest areas. The other curves are gray. PDF.

Force calls on each recorded start for Lennard-Jones 38. The box is the quartiles of those counts. A blue point is a success and a vermillion cross is a recorded failure (464). The horizontal position is the recorded count. The vertical offset only keeps marks from covering one another. PDF.

TSASE on Lennard-Jones 38. Each point is one recorded version of one method. Versions run left to right in numeric order. PAR-2 is omitted where that row has none. No line is drawn across a missing value. PDF.

eOn on Lennard-Jones 38. Each point is one recorded version of one method. Versions run left to right in numeric order. PAR-2 is omitted where that row has none. No line is drawn across a missing value. PDF.

Performance profile. Higher means more starts solved within that cost ratio. Colored lines are the six largest areas. Thin lines are the other 11.

• Sella (0.1.1) · lowest eigenvector
• rgsaddle (0.2.2) · Lanczos
• eOn (3.5.0) · Lanczos CG
• eOn (3.6.0) · Lanczos
• eOn (develop f6817b53) (3.6.0) · Lanczos
• Eon svn (r2549) · Lanczos CG

Cost distribution of 26 ranked rows. Logarithmic bins, from 17.8 to 18925.6.

Performance profile SVG Cost and success SVG

Performance profile. Select the plot for the full SVG.

Cost and success. Select the plot for the full SVG.

A shorter bar is a lower cost on a log scale shared by the codes of this problem.

Leaderboard

A shorter bar is a lower cost on the log scale for that problem. Select a column heading to sort.

Rank	Code	Best	Median	Success	PAR-2
1	Sella	17.8	17.8	100.0%	17.8
2	OPTIM	25	25	100.0%	25
3	pele	25	25	100.0%	25
4	TSASE	25	28	100.0%	25
5	rgsaddle	26.12	28.76	100.0%	26.12
6	eOn	26.385	27.3575	100.0%	26.385
7	Eon svn	26.385	28.165	100.0%	26.385
8	pARTn	2619.84	2619.84	87.0%	2619.84
9	pysisyphus	4017.14	4017.14	80.0%	4017.14
10	SoftSaddle	18700.6	18813.1	6.5%	18700.6

The tables below list every ranked row, by method family and overall.

A run short of the full set, or a run that is not the stock method, stays off the ranked table.

Minimum mode

Minimum mode. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	Sella (0.1.1) · lowest eigenvector	17.8	force calls	0	0	200
2	TSASE (410) · tsase-lanczos	25	force calls	0	0	200
3	rgsaddle (0.2.2) · Lanczos	26.12	force calls	0	0	200
4	Eon svn (r2549) · Lanczos CG	26.385	force calls	0	0	200
5	eOn (3.5.0) · Lanczos CG	26.385	force calls	0	0	200
6	eOn (3.6.0) · Lanczos	26.385	force calls	0	0	200
7	eOn (develop f6817b53) (3.6.0) · Lanczos	26.385	force calls	0	0	200
8	TSASE (408) · tsase-dimer-bfgs	27	force calls	0	0	200
9	eOn (r2156) · eon-dimer-lbfgs	28	force calls	0	0	200
10	eOn (r2156) · eon-lanczos	28	force calls	0	0	200
11	Eon svn (r2549) · Dimer L-BFGS	28.33	force calls	0	0	200
12	eOn (3.5.0) · Dimer L-BFGS	28.33	force calls	0	0	200
13	TSASE (408) · tsase-dimer-cg	29	force calls	0	0	200
14	eOn (r2156) · eon-dimer-cg	30	force calls	0	0	200
15	Eon svn (r2549) · Dimer CG	30.355	force calls	0	0	200
16	eOn (3.5.0) · Dimer CG	30.355	force calls	0	0	200
17	eOn (3.6.0) · Dimer	30.355	force calls	0	0	200
18	rgsaddle (0.2.2) · Dimer	31.4	force calls	0	0	200
19	TSASE (1.0) · SSDimer	2420.61	force calls	0	0	200
20	pARTn (1.1.0) · ARTn Lanczos	2619.84	force calls	0	0	200
21	pysisyphus (0.1.dev1) · Lanczos	4017.14	force calls	0	0	200
22	SoftSaddle (2019.04) · Lanczos	18700.6	force calls	0	0	200
23	SoftSaddle (2019.04) · Davidson	18925.6	force calls	0	0	200

Single-ended saddle

Single-ended saddle. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	Sella (0.1.1) · sella	17.8	force calls	0	0	200

Connect / double-ended

Connect / double-ended. One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Rank	Row	Cost	Unit	Hessians	FD calls	Runs
1	OPTIM · hybrid eigenvector following	25	force calls	0	0	200
2	pele · hybrid eigenvector following	25	force calls	0	0	200

Overall

One row is one code, one version, and one method. A lower cost is better. PAR-2 counts a finished run as the calls it used and a failed run as twice the cap. An archived row shows the average it published, and its PAR-2 applies that rule to the failures the entry records. Hessians is the mean analytic Hessian count. FD calls is that count times one force call per coordinate.

Row	Family	Family rank	Unit	PAR-2	Published	Median calls	Mean calls	Hessians	FD calls	Failures	Runs
Sella (0.1.1) · lowest eigenvector	Minimum mode	1	force calls	17.8		17	17.8	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
Sella (0.1.1) · sella	Single-ended saddle	1	force calls	17.8	17.8		17.8	0	0	0 failed	200
OPTIM · hybrid eigenvector following	Connect / double-ended	1	force calls	25	25	23	25	0	0	0 failed	200
TSASE (410) · tsase-lanczos	Minimum mode	2	force calls	25	25		25	0	0	0 failed	200
pele · hybrid eigenvector following	Connect / double-ended	2	force calls	25	25		25	0	0	0 failed	200
rgsaddle (0.2.2) · Lanczos	Minimum mode	3	force calls	26.12		26	26.12	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
Eon svn (r2549) · Lanczos CG	Minimum mode	4	force calls	26.385		25	26.385	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (3.5.0) · Lanczos CG	Minimum mode	5	force calls	26.385		25	26.385	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (3.6.0) · Lanczos	Minimum mode	6	force calls	26.385		25	26.385	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (develop f6817b53) (3.6.0) · Lanczos	Minimum mode	7	force calls	26.385		25	26.385	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
TSASE (408) · tsase-dimer-bfgs	Minimum mode	8	force calls	27	27		27	0	0	0 failed	200
eOn (r2156) · eon-dimer-lbfgs	Minimum mode	9	force calls	28	28		28	0	0	0 failed	200
eOn (r2156) · eon-lanczos	Minimum mode	10	force calls	28	28		28	0	0	0 failed	200
Eon svn (r2549) · Dimer L-BFGS	Minimum mode	11	force calls	28.33		27	28.33	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (3.5.0) · Dimer L-BFGS	Minimum mode	12	force calls	28.33		27	28.33	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
TSASE (408) · tsase-dimer-cg	Minimum mode	13	force calls	29	29		29	0	0	0 failed	200
eOn (r2156) · eon-dimer-cg	Minimum mode	14	force calls	30	30		30	0	0	0 failed	200
Eon svn (r2549) · Dimer CG	Minimum mode	15	force calls	30.355		29	30.355	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (3.5.0) · Dimer CG	Minimum mode	16	force calls	30.355		29	30.355	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
eOn (3.6.0) · Dimer	Minimum mode	17	force calls	30.355		29	30.355	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
rgsaddle (0.2.2) · Dimer	Minimum mode	18	force calls	31.4		23	31.4	0	0	crash 0, divergence 0, cap 0, wrong_stationary 0	200
TSASE (1.0) · SSDimer	Minimum mode	19	force calls	2420.61		233	267.945	0	0	crash 0, divergence 22, cap 0, wrong_stationary 0	200
pARTn (1.1.0) · ARTn Lanczos	Minimum mode	20	force calls	2619.84		23	23.99	0	0	crash 0, divergence 26, cap 0, wrong_stationary 0	200
pysisyphus (0.1.dev1) · Lanczos	Minimum mode	21	force calls	4017.14		22	22.195	0	0	crash 0, divergence 40, cap 0, wrong_stationary 0	200
SoftSaddle (2019.04) · Lanczos	Minimum mode	22	force calls	18700.6		19	18.91	0	0	crash 0, divergence 187, cap 0, wrong_stationary 0	200
SoftSaddle (2019.04) · Davidson	Minimum mode	23	force calls	18925.6		695	696.475	0	0	crash 0, divergence 189, cap 0, wrong_stationary 0	200

Success on this problem: harness calls under the cap and absolute overlap of at least 0.99 with the reference lowest mode.