Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ANSIENG-3858] Fix mTLS super.users in kraft upgrades #1907

Merged
merged 11 commits into from
Mar 10, 2025

Conversation

rrbadiani
Copy link
Member

@rrbadiani rrbadiani commented Feb 20, 2025

Description

Behaviour before the PR

During any kind of update to a KRaft cluster which has mTLS enabled we currently cant pass tags to run controller and broker separately and also need to give kraft and kafka super users in the custom properties of broker and controller.
Reason is during controller update phase, the super.users list is initially constructed with only the controllers' cert principals, after which the controllers are restarted. At this juncture, the broker - controller communication breaks as broker's cert principals are also needed in kraft controller super.users leading to cluster going down until we also update brokers. This breakdown is avoided by passing in super.users from custom_properties variables to kraft controller.
When we update brokers they go and fetch controller's cert principals and write to their super.users and also send their own cert principals to controllers and restart the controllers.

Solution Approach

Take the kraft and kafka cert principals from users as input via new variable rbac_mtls_kafka_kraft_cert_principals

  1. If this variable is not defined the older way of upgrade will kick in.
    1.1 Must use --tags kafka_controller,kafka_broker (both must be given)
    1.2 super.users must be defined in the custom properties in the inventory file
  2. If this new variable is defined
    1.1 This variable contains cert principals of kafka and kraft both. So while controller gets setup it doesnt lose on kafka cert principals.
    1.2 Also when broker gets setup it doesnt need to go into controller to get its mtls cert principals and send its own cert principals. Thus making both roles not dependent on each other.

Fixes # (issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

tests run1
tests run2

Manual vm based testing

  • Colocated brokers and controllers
  • Brokers and controllers on seperate VMs

Checklist:

  • Any variable/code changes have been validated to be backwards compatible (doesn't break upgrade)
  • I have added tests that prove my fix is effective or that my feature works
  • If required, I have ensured the changes can be discovered by cp-ansible discovery codebase
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules

Sorry, something went wrong.

cert principal as input and make kraft and kafka roles independently
executable thus allowing the use of --tags kafka_controller and --tags
kafka_broker seperately even when mTLS is enabled
…on kraft case
@rrbadiani rrbadiani requested a review from a team as a code owner February 20, 2025 13:23
…cy between kafka and kraft
…ovided ks or custom certs
@rrbadiani rrbadiani changed the title <WIP> [ANSIENG-3858] Fix mtls super user kraft2 [ANSIENG-3858] Fix mTLS super.users in kraft upgrades Mar 5, 2025
…put it in docs
…s per pr review comments
@airlock-confluentinc airlock-confluentinc bot force-pushed the fix-mtls-super-user-kraft2-ANSIENG-3858 branch from e6a5abb to fe3a002 Compare March 10, 2025 04:58
Copy link
Contributor

@mansi-jain-1206 mansi-jain-1206 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks!

Copy link
Member

@mansisinha mansisinha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can also consider if this variable can be made generic for all rbac cases. i.e the variable does not depend on mtls and in any of the cases where the customers need to add super users, they can simply use this variable and the values defined in this variable will be internally added to super users list along with internal values that ansible creates.
I'll add that the warning would still be present in case the customer does not specify this variable in case of mtls but this variable can be used in other cases as well. In short, good to have for other auth whereas required for mtls rbac

…take rbac super users input
…take rbac super users input for molecule tests
@rrbadiani rrbadiani merged commit db02565 into 7.4.x Mar 10, 2025
2 checks passed
@rrbadiani rrbadiani deleted the fix-mtls-super-user-kraft2-ANSIENG-3858 branch March 10, 2025 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants