Generalizing the Dual-Space Inequality to Nonsmooth Losses

Computational study of co-coercivity extensions for online convex optimization

0%
Smooth Violations
~35%
Nonsmooth Violations
500
Pairs Tested
5
Loss Types

Violation Rate Analysis

Inequality Violations by Loss Type

Moreau Envelope Smoothing

Smoothing nonsmooth losses via Moreau envelopes to restore the inequality.

Mean Ratio vs Moreau Parameter

Online Convex Optimization Regret

Average Loss

Cumulative Gradient Norm

Key Findings

Smooth: Inequality Holds

Quadratic and logistic losses satisfy the inequality perfectly (0% violations), confirming theoretical predictions.

Nonsmooth: ~35% Violations

Hinge and absolute value losses violate the inequality at non-differentiable points where subgradient selection matters.

Moreau Envelope Path

Moreau envelope smoothing restores the inequality for all nonsmooth losses with appropriate smoothing parameter.

Regret Still Well-Behaved

Despite inequality violations, OCO gradient norm regret remains controlled, suggesting weaker conditions may suffice.