Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refreshing S3 web identity token fails due to missing socket permission #119747

Closed
pxsalehi opened this issue Jan 8, 2025 · 1 comment · Fixed by #119748
Closed

Refreshing S3 web identity token fails due to missing socket permission #119747

pxsalehi opened this issue Jan 8, 2025 · 1 comment · Fixed by #119748
Labels
>bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed Coordination Meta label for Distributed Coordination team

Comments

@pxsalehi
Copy link
Member

pxsalehi commented Jan 8, 2025

In #104135, we added support for refreshing web identity credentials when the relevant file on disk is changed. However, the refresh call is not issued with sufficient permissions and it leads to the refresh call failing with:

[2024-11-23T04:27:32,004][WARN ][o.e.w.FileWatcher        ] [...] cannot notify file changes listener 
 com.amazonaws.AmazonClientException: java.security.AccessControlException: access denied ("java.net.SocketPermission" "1.2.3.4:443" "connect,resolve")
        at com.amazonaws.auth.RefreshableTask.refreshValue(RefreshableTask.java:303)
        at com.amazonaws.auth.RefreshableTask.forceGetValue(RefreshableTask.java:208)
        at com.amazonaws.auth.STSAssumeRoleWithWebIdentitySessionCredentialsProvider.refresh(STSAssumeRoleWithWebIdentitySessionCredentialsProvider.java:135)
        at org.elasticsearch.repositories.s3.S3Service$CustomWebIdentityTokenCredentialsProvider$1.onFileChanged(S3Service.java:419)
        at [email protected]/org.elasticsearch.watcher.FileWatcher$FileObserver.onFileChanged(FileWatcher.java:367)
        at [email protected]/org.elasticsearch.watcher.FileWatcher$FileObserver.checkAndNotify(FileWatcher.java:196)
        at [email protected]/org.elasticsearch.watcher.FileWatcher.doCheckAndNotify(FileWatcher.java:79)

The current workaround is to restart the affected nodes. The refresh support was added in 8.13.
@pxsalehi pxsalehi added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >bug labels Jan 8, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Distributed Coordination Meta label for Distributed Coordination team label Jan 8, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

@pxsalehi pxsalehi mentioned this issue Jan 8, 2025
Merged
pxsalehi added a commit to pxsalehi/elasticsearch that referenced this issue Jan 9, 2025
@pxsalehi
Issue S3 web identity token refresh call with sufficient permissions (e…
3b9fcef 
…lastic#119748)

Closes elastic#119747
elasticsearchmachine pushed a commit that referenced this issue Jan 9, 2025
@pxsalehi @felixbarny
Issue S3 web identity token refresh call with sufficient permissions (#…
61e16d4 
…119748) (#119840)

Closes #119747

Co-authored-by: Felix Barnsteiner <[email protected]>
pxsalehi added a commit to pxsalehi/elasticsearch that referenced this issue Jan 10, 2025
@pxsalehi
Issue S3 web identity token refresh call with sufficient permissions (e…
75d85cf 
…lastic#119748)

Closes elastic#119747
pxsalehi added a commit to pxsalehi/elasticsearch that referenced this issue Jan 10, 2025
@pxsalehi
Issue S3 web identity token refresh call with sufficient permissions (e…
01e820c 
…lastic#119748)

Closes elastic#119747
elasticsearchmachine pushed a commit that referenced this issue Jan 10, 2025
@pxsalehi
Issue S3 web identity token refresh call with sufficient permissions (#…
eb6c6fd 
…119748) (#119929)

Closes #119747
elasticsearchmachine pushed a commit that referenced this issue Jan 10, 2025
@pxsalehi
Issue S3 web identity token refresh call with sufficient permissions (#…
978eb1c 
…119748) (#119931)

Closes #119747
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed Coordination Meta label for Distributed Coordination team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Issue S3 web identity token refresh call with sufficient permissions pxsalehi/elasticsearch
2 participants
@pxsalehi @elasticsearchmachine