-
Notifications
You must be signed in to change notification settings - Fork 662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Idempotence tests incorrectly succeed when Ansible callback plugin is not default #4063
Comments
Out of interest does setting |
It does not in this case, as the format of the
I think that env var controls output format from tasks themselves rather than the recap, and as molecule is looking for the lines from the recap, whatever gets printed by the task itself shouldn't matter (unless you happen to print |
I PR would be welcomed here. The right way to address this should be to enforce use of a specific callback (included in core), so we can analyze the output. |
RFC: Use the |
Prerequisites
pip check
does not report any conflictsEnvironment
❯ python3 --version
Python 3.11.2
What happened
What happened?
We have CI that uses molecule to automatically test pull requests to our internal company repositories of Ansible roles. We discovered that idempotent tests were showing as passed in CI, but when locally they would fail. This was narrowed down to the difference being that locally, we use the default
ansible.builtin.default
callback, while we useansible.builtin.junit
callback in CI.What did we expect to happen?
We expected tests to fail in CI as they were locally.
Root cause?
I believe this is the root cause. In
idempotence.py
, we have the following function:molecule/src/molecule/command/idempotence.py
Lines 62 to 78 in 01631c7
Specifically, the following line:
molecule is detecting the changed_when status from the ansible output by specifically looking for the
changed=[some_number]
format that exists in the default callback. However, with other callback plugins Ansible doesn't print those lines and instead uses other formats. For example, inansible.builtin.json
we instead get:I'm happy to put in PRs for this, but I'd like to hear what the maintainers think about how to restructure this function. Writing a regex per parser seems highly undesirable and a pain to maintain. String parsing doesn't seem like the most ideal way to go about this, but because there's so many different callback plugin types that might be unavoidable.
Reproducing example
Gist here: https://gist.github.com/chrwang/bad5de8f073bd6a8cedc1332f2737768
Command run for success:
Command(s) run for failure:
The text was updated successfully, but these errors were encountered: