Permissions required for stateful agentless integrations #118644
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elasticsearchmachine
added
v9.0.0
external-contributor
|
@orestisfl please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation. |
orestisfl
added
the
auto-backport
orestisfl
changed the title
9 tasks
elasticsearchmachine
added
the
needs:triage
astefan
added
:Security/Authorization
|
Pinging @elastic/es-security (Team:Security) |
orestisfl
added a commit
to elastic/kibana
that referenced
this pull request
Jan 7, 2025
## Summary Adds necessary permissions to write to the `agentless-*` index. See: - Elasticsearch PR: elastic/elasticsearch#118644 - Context: elastic/security-team#11104 As part of elastic/security-team#11104, we need to write integration data that needs to be persistent. The implementation we are working on, uses Elasticsearch as the storage mechanism for this data. Normally, integrations write to data streams instead of normal ES indices. However, data streams cannot provide a generic implementation for our use case and thus we need a normal ES index. This PR grants permissions from the fleet service account to the agentless integrations to write to `agentless-*` ES indices. In `x-pack/plugins/fleet/server/services/agent_policies/package_policies_to_agent_permissions.ts` there are other examples of other integrations that need ES index permissions so there is prior art in doing this. The difference with this PR however, is that we need to conditionally merge the extra `agentless-*` permissions with any potential existing data stream permissions since we are dealing with arbitrary agentless integrations.
jakelandis
requested changes
Jan 7, 2025
There was a problem hiding this comment.
It looks like the CI failure is related and will need update https://github.com/elastic/elasticsearch/blob/main/docs/reference/rest-api/security/get-service-accounts.asciidoc to account for the new privilege in response. Should be able to test and reproduce locally with ./gradlew ":docs:yamlRestTest" --tests "org.elasticsearch.smoketest.DocsClientYamlTestSuiteIT.test {yaml=reference/rest-api/security/get-service-accounts/line_57}" (not tested)
Also, ideally these indindes would be hidden, which requires an index template or explicit creation ; but that is orthogonal to the privileges being introduced here.
kowalczyk-krzysztof
pushed a commit
to kowalczyk-krzysztof/kibana
that referenced
this pull request
Jan 7, 2025
## Summary Adds necessary permissions to write to the `agentless-*` index. See: - Elasticsearch PR: elastic/elasticsearch#118644 - Context: elastic/security-team#11104 As part of elastic/security-team#11104, we need to write integration data that needs to be persistent. The implementation we are working on, uses Elasticsearch as the storage mechanism for this data. Normally, integrations write to data streams instead of normal ES indices. However, data streams cannot provide a generic implementation for our use case and thus we need a normal ES index. This PR grants permissions from the fleet service account to the agentless integrations to write to `agentless-*` ES indices. In `x-pack/plugins/fleet/server/services/agent_policies/package_policies_to_agent_permissions.ts` there are other examples of other integrations that need ES index permissions so there is prior art in doing this. The difference with this PR however, is that we need to conditionally merge the extra `agentless-*` permissions with any potential existing data stream permissions since we are dealing with arbitrary agentless integrations.
orestisfl
force-pushed
the
agentless-storage-permissions
branch
from
3667d0a to
c4eb0fa
Compare
orestisfl
force-pushed
the
agentless-storage-permissions
branch
from
c4eb0fa to
db8f1f6
Compare
|
@jakelandis managed to fix the tests and introduced the index templates, could you please take another look? |
jakelandis
approved these changes
Jan 9, 2025
| "index": { | ||
| "hidden": true, | ||
| "number_of_shards": 1, | ||
| "number_of_replicas": 1 |
There was a problem hiding this comment.
nitpick: I would suggest to omit these (shard/replica) as they are the defaults.
💔 Backport failed
You can use sqren/backport to manually backport by running |
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
orestisfl
added a commit
to orestisfl/elasticsearch
that referenced
this pull request
Jan 10, 2025
) Closes elastic/security-team#11102 Closes elastic/security-team#11104 This allows agentless integrations (via elastic/beats#41446, elastic/kibana#203810) to write to agentless-* indices. Each index is created on-demand by the filebeat client and kibana conditionally extends the API key permissions to allow writing to the index. (cherry picked from commit 3c184b9) # Conflicts: # docs/reference/rest-api/security/get-service-accounts.asciidoc # x-pack/plugin/security/qa/service-account/src/javaRestTest/java/org/elasticsearch/xpack/security/authc/service/ServiceAccountIT.java # x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/service/ElasticServiceAccounts.java
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes https://github.com/elastic/security-team/issues/11102
Closes https://github.com/elastic/security-team/issues/11104
This allows agentless integrations (via elastic/beats#41446, elastic/kibana#203810) to write to
agentless-*indices. Each index is created on-demand by the filebeat client and kibana conditionally extends the API key permissions to allow writing to the index.