Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Log warning if yarn workstreams foreach attempts to execute multiple jobs but that is not possible given the available resources and the --jobs options in not set #6669

Open
1 task done
jordanpagewhite opened this issue Jan 29, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@jordanpagewhite
Copy link

  • I'd be willing to implement this feature (contributing guide)
  • [?] This feature is important to have in this repository; a contrib plugin wouldn't do

It seems to me that it would be ideal to have this in foreach.ts, but I'm 100% open to / encourage a discussion about the pros/cons of other approaches.

Describe the user story

As a developer, I would like to be able to execute yarn dev, where yarn dev is (example) yarn workspaces foreach -A --include backend --include app --parallel -v -i run start with 2 or more workspaces that must run in parallel

AND

I am running in a container (or other environment) where there are 2 or less CPU cores available

AND GIVEN

If -p,--parallel is set, the commands will be ran in parallel; they'll by default be limited to a number of parallel tasks roughly equal to half your core number, but that can be overridden via -j,--jobs, or disabled by setting -j unlimited.

Source: https://yarnpkg.com/cli/workspaces/foreach

THEN

I would like a warning to be logged to stdout or stderr informing me that it is impossible for the number of parallel tasks I've tried to start in my yarn dev to be executed given the CPU cores available in this environment

Describe the solution you'd like

Given the user story above, I would like a warning to be logged to stdout or stderr informing me that it is impossible for the number of parallel tasks I've tried to start in my yarn dev to be executed given the CPU cores available in this environment.

In foreach.ts and before we execute pLimit, could we check the available parallelism using os's availableParallelism() and determine if we have the number of CPU cores available we would need to execute the number of parallel tasks that have been dictated in the yarn workspaces foreach command?

If, for any reason, we cannot confidently determine the number of available CPU cores via availableParallelism(), we could use a conditional and not log a warning / maintain the exact same behavior the command has today.

Describe the drawbacks of your solution

I am not aware of drawbacks, but I am hoping we can discuss those in this issue.

Describe alternatives you've considered

I have not explored turning this into a plugin do to the nature of 'where' this code would have to be executed within the foreach command's execution. I am, however, open to any ideas anyone may have to solve this in a different way.

@jordanpagewhite jordanpagewhite added the enhancement New feature or request label Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant