Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Widen options for momentum for our APGD class #2060

Open
MargaretDuff opened this issue Feb 3, 2025 · 0 comments · May be fixed by #2061
Open

Widen options for momentum for our APGD class #2060

MargaretDuff opened this issue Feb 3, 2025 · 0 comments · May be fixed by #2061
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@MargaretDuff
Copy link
Member

Description

For general convex cases (non-smooth, inexact regularisers), we cannot use the current implementation of FISTA combined with stochastic estimators. For strongly convex objective we can run FISTA+SVRG but we need another choice of momentum.

As suggested by @epapoutsellis, we could change the update step in APGD to

Image

Where momentum is like a step_size_method or callback and has access to the whole algorithm class.

To get FISTA you would set the step size method to 1/L and choose the momentum based on nesterov. A constant momentum would also be useful.

@MargaretDuff MargaretDuff added the enhancement New feature or request label Feb 3, 2025
@MargaretDuff MargaretDuff linked a pull request Feb 3, 2025 that will close this issue
11 tasks
@github-project-automation github-project-automation bot moved this to Todo in CIL work Feb 6, 2025
@MargaretDuff MargaretDuff moved this from Todo to Blocked in CIL work Feb 6, 2025
@MargaretDuff MargaretDuff added this to the v25.0.0 milestone Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Blocked
Development

Successfully merging a pull request may close this issue.

2 participants