You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
It is not immediately obvious that aleph and ingest-file workers are using "number of cpus" threads, which can cause scaling issues as seen in numerous reports, the last of which is #3383 .
To Reproduce
Steps to reproduce the behavior:
Follow the installation / deployment docs
Massively scale up ingest-file, workers etc.
Run out of postgres connections
Expected behavior
Have better control over the number of workers running. Make it visible that we are using threads by default. Explain what the servicelayer workers default to and how to disable that.
Aleph version
3.15.1
Screenshots
Additional context
The text was updated successfully, but these errors were encountered:
stchris
added
bug
Things that should work, but don’t
docs
Issues related to Aleph’s documentation
triage
These issues need to be reviewed by the Aleph team
labels
Oct 5, 2023
Would be good to set the limit to 0, and disable threads entirely. Also update the documentation to talk more about how we (don't) use this feature and what we recommend other people do.
Describe the bug
It is not immediately obvious that aleph and ingest-file workers are using "number of cpus" threads, which can cause scaling issues as seen in numerous reports, the last of which is #3383 .
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Have better control over the number of workers running. Make it visible that we are using threads by default. Explain what the servicelayer workers default to and how to disable that.
Aleph version
3.15.1
Screenshots
Additional context
The text was updated successfully, but these errors were encountered: