Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] RankEvalRequestTests testSerialization failing #119859

Closed
elasticsearchmachine opened this issue Jan 9, 2025 · 6 comments · Fixed by #119875
Closed

[CI] RankEvalRequestTests testSerialization failing #119859

elasticsearchmachine opened this issue Jan 9, 2025 · 6 comments · Fixed by #119875
Assignees
Labels
:Data Management/Data streams Data streams and their lifecycles needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Data Management Meta label for data/management team >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ":modules:rank-eval:test" --tests "org.elasticsearch.index.rankeval.RankEvalRequestTests.testSerialization" -Dtests.seed=143504D5D0FD2FB9 -Dbuild.snapshot=false -Dtests.jvm.argline="-Dbuild.snapshot=false" -Dlicense.key=x-pack/license-tools/src/test/resources/public.key -Dtests.locale=kxv-Telu-IN -Dtests.timezone=America/Cuiaba -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: <org.elasticsearch.index.rankeval.RankEvalRequest/unset>
     but: was <org.elasticsearch.index.rankeval.RankEvalRequest/unset>

Issue Reasons:

  • [main] 3 consecutive failures in step release-tests
  • [main] 3 failures in test testSerialization (1.2% fail rate in 248 executions)
  • [main] 3 failures in step release-tests (60.0% fail rate in 5 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

@elasticsearchmachine elasticsearchmachine added :Search Relevance/Ranking Scoring, rescoring, rank evaluation. >test-failure Triaged test failures from CI labels Jan 9, 2025
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 3 consecutive failures in step release-tests
  • [main] 3 failures in test testSerialization (1.2% fail rate in 248 executions)
  • [main] 3 failures in step release-tests (60.0% fail rate in 5 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Jan 9, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@benwtrent
Copy link
Member

Something strange indeed is going on. There are many (seemingly unrelated tests) all failing with this same sort of failure.

See: #119822

@benwtrent
Copy link
Member

Git bisect says: 82b1f2a

But I don't believe that. Something else is going on.

@benwtrent
Copy link
Member

OK, gitbisect was off by one, it appears to be: c3839e1. It fails on many different seeds with that commit.

@jbaiera I am gonna assign this issue and the others to y'all. There's something strange in the neighborhood.

@benwtrent benwtrent added :Data Management/Data streams Data streams and their lifecycles and removed :Search Relevance/Ranking Scoring, rescoring, rank evaluation. labels Jan 9, 2025
@elasticsearchmachine elasticsearchmachine added Team:Data Management Meta label for data/management team and removed Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Jan 9, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-data-management (Team:Data Management)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Data streams Data streams and their lifecycles needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Data Management Meta label for data/management team >test-failure Triaged test failures from CI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants