Transient objective convergence of spectrally weighted gradient descent

Summary in plain English

An optimization method can be good at shrinking the quantity it watches while making slower progress on the quantity we actually care about. This paper demonstrates that distinction for a familiar family of gradient methods applied to positive-definite quadratic problems.

The methods choose their step using a weighted gradient. The paper measures the ordinary objective gap: how far the current function value lies above its minimum. Those are different yardsticks.

A three-variable example makes the difference concrete. After two steps of standard minimal-gradient descent, about 93.61% of the initial objective gap remains. A proposed two-step application of the steepest-descent rate would allow at most 92.31%. Exact fractions, rather than rounded decimals, establish the violation.

The main result goes beyond that example. It identifies the sharp leading worst-case objective decrease at a fixed iteration horizon as the condition number grows, and controls longer horizons for power weights.

This is an AI-assisted, unrefereed mathematical candidate. It studies established algorithms; it does not introduce a new optimizer or refute the originating paper's proved convergence results.

Summary for specialists

For $A=A^T\succ0$, let

The update is

with

Normalize $\lambda_{\min}(A)=1$ and $\lambda_{\max}(A)=\kappa$. For a general nonhomogeneous weight, this spectral scale is part of the statement. If $\Psi$ is fixed, positive, nondecreasing and unbounded, then for every fixed integer $K\ge1$ and $0<\omega\le1$, the worst-case objective ratio satisfies

Three dimensions suffice; two suffice when $\omega<1$.

For power weights $\Psi(\lambda)=\lambda^{2\ell}$, fixed $\ell>0$ and $\omega=1$, the coefficient remains sharp for every integer-valued $K=o(\log\kappa)$. A separate family gives simultaneous violation of the per-iteration Kantorovich bound for all

once $\kappa$ is sufficiently large. One instance at each condition number works throughout this interval.

Technical account

Which two-step claim fails?

Write

Two readings of a recent prose question have different answers. For unrelaxed monotone weights, one factor over two steps is valid:

Two factors over the same two steps need not be:

The explicit minimal-gradient counterexample uses

Its exact objective ratio is

This exceeds $(99/101)^4\approx0.92311388$. The paper supplies the exact positive difference. A second rational example has condition number ten.

Why three modes?

A small high-frequency component can dominate the weighted step calculation while contributing little ordinary objective energy. Two high modes keep the step short; a low mode retains most of the objective gap.

With only two active eigenvalues and no relaxation, the weighted spectral ratio is inverted at each step. Two consecutive updates therefore multiply both error components by the same scalar, which enforces the stronger two-step bound. Three active eigenvalues are the smallest support permitting this unrelaxed failure.

The underlying weighted-gradient family and two-mode dynamics are classical. The new contribution claimed here is the objective-specific transient construction and its quantitative control.

What does “finite horizon” mean?

The fixed-horizon theorem holds the number of steps fixed while the condition number increases. The growing-horizon theorem allows a prescribed sublogarithmic number of steps, with the constructed instance depending on that horizon. The simultaneous-interval result is stronger in a different direction: one instance works at every time in its stated interval.

None fixes one matrix and then follows it indefinitely. These conclusions therefore coexist with established fixed-matrix asymptotic convergence.

Lemma 6 supplies an explicit finite-parameter error estimate. The paper also rearranges its hypothesis into a usable horizon test and derives a sufficient strict-failure test. Neither condition is claimed necessary or optimal.

Evidence, assurance and limitations

The universal results rest on written proofs. Producer-side checking reproduces three rational examples, 80 two-mode cases and 324 exact recurrence comparisons. A separate Decimal grid checks 195 admissible parameter combinations; 125 combinations are excluded by the lemma's hypothesis, not counted as passes.

The grid is a numerical diagnostic, not certified interval arithmetic. Sixteen deliberate corruptions test named scientific guards in normal and optimized Python execution. This improves executable assurance without converting finite checks into a proof of the asymptotic theorems.

No external human peer review, independent external reproduction or proof-assistant formalisation is claimed.

The paper does not identify the exact worst-case ratio at every finite condition number, a sharp logarithmic crossover, or a lower bound for arbitrary optimization algorithms. The moving-horizon results are restricted to positive power weights without relaxation. A factor-two difference in a transient coefficient does not imply twice as many iterations to attain a fixed accuracy.

Relationship to earlier work

MacDonald, Murray and Tappenden's published Section 6 asks a two-step question in prose. This paper separates its two precise readings rather than treating either as a previously proved theorem.

Pronzato, Wynn and Zhigljavsky, and Huang and collaborators, establish weighted-gradient and fixed-matrix asymptotic structure. He's GMRES(1) analysis concerns residual and root-convergence factors. The later MacDonald–Murray–Tappenden flexible framework includes multiple search directions and symmetric preconditioning, with guarantees in a selected weighted gradient norm.

Those distinctions matter: ordinary objective error, weighted gradient error, and fixed-matrix long-time rates are not interchangeable. The package's source comparison gives the inspected versions and exact passages. Its bounded search found no matching full transient result, but cannot guarantee historical priority.

Who should care, and why

Why the problem matters

A convergence guarantee is only useful when its error metric and order of limits match the question being asked. Here, three dimensions expose a gap that two-mode tests cannot reveal. The result supplies both a simple exact counterexample and a sharp description of a broader transient regime.

How to inspect or reproduce the recorded checks

Start with the paper's metric/regime table and Section 6's exact examples. Theorem 3 gives the fixed-horizon result; Lemma 6 and its following finite-parameter test explain the quantitative control; Theorem 7 and Corollary 8 handle growing horizons.

Read the package's verification record before running the standard-library checkers. The exact-example, horizon-grid and intended-failure scripts require a fresh output path. Compare exact fractions separately from Decimal diagnostics. Repeatability of these finite checks does not establish the universal theorem or external independence.

The most valuable next projects

A sharp transition when the horizon is comparable to $\log\kappa$ would clarify how the transient joins the established long-time picture. Over-relaxation and bounded spectral weights require separate analysis. Merely collecting more examples of the existing failure would chiefly strengthen assurance rather than resolve those questions.

What is in the evidence package

The package includes the self-contained paper in PDF, LaTeX and accessible Markdown, the exact examples and checkers, a claim-to-proof reading guide, source comparison, limitations, licence map and substantive revision response. Private editorial assessments are not required to understand or reproduce the public argument.

@misc{weightedgradienttransientobjective2026,
  title        = {Transient objective convergence of spectrally weighted gradient descent},
  author       = {Anonymous},
  year         = {2026},
  doi          = {10.5281/zenodo.24001734},
  url          = {https://doi.org/10.5281/zenodo.24001734},
  version      = {0.1.0-candidate},
  howpublished = {Zenodo},
  note         = {Unrefereed; internally replayed evidence package. Press page: https://evidencepress.org/releases/weighted-gradient-transient-objective/}
}