Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[poc] SLO operation health rule #211508

Closed
wants to merge 5 commits into from

Conversation

kdelemme
Copy link
Contributor

@kdelemme kdelemme commented Feb 17, 2025

POC introducing a rule for monitoring SLO operational health
alert

Performance testing the rule with 100 SLOs using labels.projectIds as groupBy. This is using ephemeral projects from data-forge: --ephemeral-project-ids = 10 --events-by-cycle = 100

1 rule is taking about 1s to run when the system stabilized. But was taking 40s when the system was under heavy load with SLO transforms being created.
image

@kdelemme kdelemme added release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-management Observability Management User Experience Team labels Feb 17, 2025
@elasticmachine
Copy link
Contributor

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!

@kdelemme kdelemme force-pushed the poc/slo-health-rules branch from 333962e to 5399653 Compare February 18, 2025 15:10
@kdelemme
Copy link
Contributor Author

Decision has been made to not rely on a rule to monitor stack health. cc @jasonrhodes

@kdelemme kdelemme closed this Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-management Observability Management User Experience Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants