Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] OperatorPrivilegesIT testEveryActionIsEitherOperatorOnlyOrNonOperator failing #118220

Closed
elasticsearchmachine opened this issue Dec 8, 2024 · 3 comments · Fixed by #119386
Closed

[CI] OperatorPrivilegesIT testEveryActionIsEitherOperatorOnlyOrNonOperator failing #118220

elasticsearchmachine opened this issue Dec 8, 2024 · 3 comments · Fixed by #119386
Assignees
@n1v0lg
Labels
low-risk An open issue or test failure that is a low risk to future releases :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Dec 8, 2024 •
edited
Loading

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:security:qa:operator-privileges-tests:javaRestTest" --tests "org.elasticsearch.xpack.security.operator.OperatorPrivilegesIT.testEveryActionIsEitherOperatorOnlyOrNonOperator" -Dtests.seed=5F49A956EFD585D7 -Dbuild.snapshot=false -Dtests.jvm.argline="-Dbuild.snapshot=false" -Dlicense.key=x-pack/license-tools/src/test/resources/public.key -Dtests.locale=agq-CM -Dtests.timezone=America/Whitehorse -Druntime.java=23

Applicable branches:
8.x

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Actions may no longer be valid: [[cluster:monitor/xpack/inference/unified]]. They should be removed from either the operator-only action registry in [org.elasticsearch.xpack.security.operator.DefaultOperatorOnlyRegistry] or the non-operator action list in [org.elasticsearch.xpack.security.operator.Constants]

Issue Reasons:

  • [8.x] 16 consecutive failures in step release-tests
  • [8.x] 16 failures in test testEveryActionIsEitherOperatorOnlyOrNonOperator (2.4% fail rate in 675 executions)
  • [8.x] 16 failures in step release-tests (100.0% fail rate in 16 executions)
  • [8.x] 16 failures in pipeline elasticsearch-periodic (84.2% fail rate in 19 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC >test-failure Triaged test failures from CI labels Dec 8, 2024
elasticsearchmachine added a commit that referenced this issue Dec 8, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.security.operator.OperatorPrivilegesIT t…
4a8a8a0 
…estEveryActionIsEitherOperatorOnlyOrNonOperator #118220
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 9 failures in test testEveryActionIsEitherOperatorOnlyOrNonOperator (1.0% fail rate in 911 executions)
  • [main] 8 failures in step part-4 (1.9% fail rate in 412 executions)
  • [main] 8 failures in pipeline elasticsearch-pull-request (1.9% fail rate in 411 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added Team:Security Meta label for security team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Dec 8, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-security (Team:Security)

@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 2 consecutive failures in step release-tests
  • [main] 6 failures in test testEveryActionIsEitherOperatorOnlyOrNonOperator (0.9% fail rate in 645 executions)
  • [main] 2 failures in step release-tests (16.7% fail rate in 12 executions)
  • [main] 4 failures in step part-4 (1.4% fail rate in 291 executions)
  • [main] 2 failures in pipeline elasticsearch-periodic (15.4% fail rate in 13 executions)
  • [main] 4 failures in pipeline elasticsearch-pull-request (1.4% fail rate in 290 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Dec 11, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.security.operator.OperatorPrivilegesIT t…
d5589d7 
…estEveryActionIsEitherOperatorOnlyOrNonOperator #118220
@n1v0lg n1v0lg added low-risk An open issue or test failure that is a low risk to future releases and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels Dec 13, 2024
@n1v0lg n1v0lg mentioned this issue Dec 31, 2024
Merged
elasticsearchmachine pushed a commit that referenced this issue Jan 7, 2025
@n1v0lg
Skip operator privilege action test on release builds (#119386)
5f9a9e0 
The `testEveryActionIsEitherOperatorOnlyOrNonOperator` test has been
reliably muted because release builds frequently miss actions due to
disabled feature flags. There's no straight-forward way to account for
this. This PR changes the test to only run on snapshot builds. This way
we get test coverage while reducing maintenance burden. 

Closes: #118220
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@n1v0lg n1v0lg

Labels
low-risk An open issue or test failure that is a low risk to future releases :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Skip operator privilege action test on release builds n1v0lg/elasticsearch
2 participants
@n1v0lg @elasticsearchmachine