Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Estimated number of threads is only valid for workers #491

Open
ThomasCrambert opened this issue Jan 20, 2025 · 3 comments
Open

Estimated number of threads is only valid for workers #491

ThomasCrambert opened this issue Jan 20, 2025 · 3 comments

Comments

@ThomasCrambert
Copy link

Hello 👋,

The check estimated_number_of_threads assumes a worker will always be started. In case the configuration specifies to boot only a scheduler or a dispatcher, the estimation will return three (main + heartbeat + threads for each worker) instead of two (heartbeat + dispatcher or heartbeat + scheduler).

Maybe the problems comes from the default value for thread_count, it is 1 when it should be 0 if no workers are specified in the configuration, WDYT?

@rosa
Copy link
Member

rosa commented Jan 20, 2025

I think the supervisor might need 3 connections as well: one for the main thread, one for the process maintenance task and one for its own heartbeat. The dispatcher, with default configuration, too: one for the polling, one for concurrency maintenance, and one for its heartbeat. I think 3 is a safe assumption so that you don't run out of DB connections, which is the purpose of this check.

@ThomasCrambert
Copy link
Author

I think 3 is a safe assumption so that you don't run out of DB connections, which is the purpose of this check.

Fair enough 👍, this function might need a rewrite tho, when we looked into it we felt it was too focused on workers.

On an other note, does your explanation also apply to the scheduler? I'm unsure it would need at least three (main + heartbeat + schedule_recurring_tasks) or if it depends on the number of recurring tasks to process at once (so main + heartbeat + n * schedule_recurring_tasks)

@rosa
Copy link
Member

rosa commented Jan 20, 2025

No, the scheduler should have enough with 2 (heartbeat + task scheduling).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants