Implement worker auto-scaler (stability and performance boost) - #6256
Open
AJenbo wants to merge 1 commit into
Open
Implement worker auto-scaler (stability and performance boost)#6256AJenbo wants to merge 1 commit into
AJenbo wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This implements worker auto scaling. Rather then simply setting a fixed number of 8 workers this looks at the available CPU threads and RAM to determin the best number of active workers.
This has two consequences:
On a co-workers memory strained MacBook
maximumNumberOfProcessesnormally has to be set to4or PHPStan will crash. With this solution PHPStan now limits it self to the available resources with out needing manual tweaking.On my workstation with 32 available threads it results in a 30% performance lift by better utilizing the available threads instead of being caped to 8 cores.
For CI where there can be a discrepancy between reported cores and available cores this implementation looks at the cgroup and CFS to determin effective limits avoiding the issues that the fixed
8default was trying to be a happy medium for.The estimated worker memory usage is based on usage on 25 real projects (application based on WordPress, PHPBB, Laravel, micro frameworks, OpenCart, Phpactor, phpstan-src etc).
I would say it's fairly well tested, but it's probably also advised to do further testing on diverse systems, for one thing I don't have access to Windows or macOS.