Skip to content

Bug: Self hosted security syncs should not re-fetch security prices one each sync #2076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ph-Os opened this issue Apr 10, 2025 · 3 comments
Open

Bug: Self hosted security syncs should not re-fetch security prices one each sync #2076

ph-Os opened this issue Apr 10, 2025 · 3 comments
Labels
💻 Self Hosted only Issues pertaining to self-hosted versions of Maybe 3️⃣ Low priority Contributions accepted, but Maybe team will not be working on this in the near term

Comments

@ph-Os
Copy link

ph-Os commented Apr 10, 2025 •
edited
Loading

How are you using Maybe?

  • I am a self-hosted user

Self hoster checklist

  • v0.4.3
  • Unraid

Bug description

  • Manually tried to add some securitites (Stocks)
  • Noticed that pop up window disappeared sometimes when loosing focus, reopenend window to input data.
  • Noticed suddenly overall very sluggish performance of maybe, blue progress bar on top moving slowly.
  • Openend live logs, maybe constantly looking for securities. See excerpt below.
  • Noticed that I ran out of 1000 free synth credits already, despite beginning of the month and low number of securities in portfolio.
  • High CPU load

I, [2025-04-10T13:49:51.123352 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-23
I, [2025-04-10T13:49:51.125891 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-23
I, [2025-04-10T13:49:51.125928 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-23
I, [2025-04-10T13:49:51.270579 #1] INFO -- : [ActiveJob] [SyncJob] [5719ee94-fdd1-4409-ab6c-cc1e3e2a2c43] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-24
I, [2025-04-10T13:49:51.272696 #1] INFO -- : [ActiveJob] [SyncJob] [5719ee94-fdd1-4409-ab6c-cc1e3e2a2c43] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-24
I, [2025-04-10T13:49:51.272742 #1] INFO -- : [ActiveJob] [SyncJob] [5719ee94-fdd1-4409-ab6c-cc1e3e2a2c43] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-24
I, [2025-04-10T13:49:51.370567 #1] INFO -- : [ActiveJob] [SyncJob] [996a1702-a34a-4ad3-aed0-6c499910aa54] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-16
I, [2025-04-10T13:49:51.372447 #1] INFO -- : [ActiveJob] [SyncJob] [996a1702-a34a-4ad3-aed0-6c499910aa54] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-16
I, [2025-04-10T13:49:51.372487 #1] INFO -- : [ActiveJob] [SyncJob] [996a1702-a34a-4ad3-aed0-6c499910aa54] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-16
I, [2025-04-10T13:49:51.434252 #1] INFO -- : [ActiveJob] [SyncJob] [801bb61e-5a7a-4c6a-b22b-364f463b89bd] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-09
I, [2025-04-10T13:49:51.436200 #1] INFO -- : [ActiveJob] [SyncJob] [801bb61e-5a7a-4c6a-b22b-364f463b89bd] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-09
I, [2025-04-10T13:49:51.437001 #1] INFO -- : [ActiveJob] [SyncJob] [801bb61e-5a7a-4c6a-b22b-364f463b89bd] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-09
I, [2025-04-10T13:49:51.627201 #1] INFO -- : [ActiveJob] [SyncJob] [54a218a8-7783-4cb3-906d-53152f7538a8] [HoldingCalculator] Generating holdings for 3 securities on 2010-10-28
I, [2025-04-10T13:49:51.629024 #1] INFO -- : [ActiveJob] [SyncJob] [54a218a8-7783-4cb3-906d-53152f7538a8] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-10-28
I, [2025-04-10T13:49:51.629064 #1] INFO -- : [ActiveJob] [SyncJob] [54a218a8-7783-4cb3-906d-53152f7538a8] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-10-28
I, [2025-04-10T13:49:51.759371 #1] INFO -- : [ActiveJob] [SyncJob] [28ee122e-93a2-4b9c-baa2-ee8ef65b4de7] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-30
I, [2025-04-10T13:49:51.761936 #1] INFO -- : [ActiveJob] [SyncJob] [28ee122e-93a2-4b9c-baa2-ee8ef65b4de7] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-30
I, [2025-04-10T13:49:51.765588 #1] INFO -- : [ActiveJob] [SyncJob] [28ee122e-93a2-4b9c-baa2-ee8ef65b4de7] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-30
I, [2025-04-10T13:49:51.904328 #1] INFO -- : [ActiveJob] [SyncJob] [d6c82406-873e-4249-99d6-0bac48ed860f] [HoldingCalculator] Generating holdings for 3 securities on 2009-12-17
I, [2025-04-10T13:49:51.906201 #1] INFO -- : [ActiveJob] [SyncJob] [d6c82406-873e-4249-99d6-0bac48ed860f] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2009-12-17
I, [2025-04-10T13:49:51.906236 #1] INFO -- : [ActiveJob] [SyncJob] [d6c82406-873e-4249-99d6-0bac48ed860f] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2009-12-17
I, [2025-04-10T13:49:51.978107 #1] INFO -- : [ActiveJob] [SyncJob] [d21d4f67-e083-43b0-90a8-f8f0964fbc71] [HoldingCalculator] Generating holdings for 3 securities on 2010-12-02
I, [2025-04-10T13:49:51.980298 #1] INFO -- : [ActiveJob] [SyncJob] [d21d4f67-e083-43b0-90a8-f8f0964fbc71] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-12-02
I, [2025-04-10T13:49:51.980341 #1] INFO -- : [ActiveJob] [SyncJob] [d21d4f67-e083-43b0-90a8-f8f0964fbc71] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-12-02
I, [2025-04-10T13:49:52.310011 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] Generating holdings for 3 securities on 2010-11-24
I, [2025-04-10T13:49:52.316490 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] No price found for security 48c57d9c-39a8-4aa3-8c36-08bb9d13b2d7 on 2010-11-24
I, [2025-04-10T13:49:52.317195 #1] INFO -- : [ActiveJob] [SyncJob] [92c0f8a2-c37c-467f-951e-901099833725] [HoldingCalculator] No price found for security 565fcd45-d1a3-4d2e-8c73-30219dcce06c on 2010-11-24

Expected behavior

What is the intended behavior that you would expect?

  • It is unclear what is going on - Is this a bug where trying to fetch unavailable security information leads to running out of synth credits ?
  • If synth credits are used up, why is the app still trying to fetch ? Like this it adds strain to my server CPU for no reason and I was forced to switch off the Maybe docker.
  • Is security info fetched at a reasonable rate or is this designed to run out at the beginning of the month for moderate use ?
  • How can I recover my already input data ( no export functionality available ) from this instance or fix this issue ?
@zachgoll zachgoll changed the title Bug: Maybe stuck in never ending sync, 1000 synth credits used up while adding securities to portfolio manually Bug: Self hosted security syncs should not re-fetch security prices one each sync Apr 23, 2025
@zachgoll zachgoll added 3️⃣ Low priority Contributions accepted, but Maybe team will not be working on this in the near term 💻 Self Hosted only Issues pertaining to self-hosted versions of Maybe labels Apr 23, 2025
@zachgoll
Copy link
Collaborator

By default, on every sync (which happens at login 1x daily OR when an account's data changes) we fetch all the required security prices needed to accurately display the account's historical graph.

Currently, we're not checking whether those prices already exist and we're upserting the differences. This is intended behavior on the hosted version of Maybe, but will deplete Synth credits on self hosted versions quicker than expected.

We should conditionally be checking whether family.self_hoster? is true and changing the behavior based on that condition.

@Senanfurkan
Copy link

Senanfurkan commented May 11, 2025 •
edited
Loading

@zachgoll The app is not usable because of this bug. I think you should re-evaluate the priority.

After adding an account with a different currency than the main one, it constantly tries to sync (like in an endless while loop) and you can not click anywhere else in UI due to the little popup appearing on top - Syncing account - . Also it used 1000 free synth credits in less then 3 minutes.

It starts as soon as I run the docker. It doesn't even need me to open the app or login

@Spamouille
Copy link

Spamouille commented May 12, 2025 •
edited
Loading

Same issue here on all of my accounts (even checking or savings), unable to add a transaction because of the constant refresh. (transaction box close itself)

Also the graph always revert to 30D or 90D after less than 3 sec.

lots of sync job in the worker log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💻 Self Hosted only Issues pertaining to self-hosted versions of Maybe 3️⃣ Low priority Contributions accepted, but Maybe team will not be working on this in the near term
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ph-Os @Spamouille @zachgoll @Senanfurkan