Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add pytest option to apply custom CPU template to any microVM #4975

Merged

Conversation

zulinx86
Copy link
Contributor

@zulinx86 zulinx86 commented Dec 30, 2024

Reason / Changes

Sometimes we would like to test a custom CPU template for debugging / testing purposes. The new pytest option --custom-cpu-template allows us to apply the given CPU template to any microVM in any test unless it is overwritten by a test. Since cpu_template_any yields not only CPU templates but also None, the given CPU template will be used in the None case.

Testing

If a bad CPU template is passed via the --custom-cpu-template, tests where another CPU template is not given failed, meaning the bad template is applied correctly.

$ sudo tools/devtool test -- integration_tests/functional/test_cpu_all.py -k test_all_vcpus_online --custom-cpu-template bad_cpu_template.json
// snipped
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-5.10.225-NO_CPU_TMPL-32] FAILED [  6%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-5.10.225-static_V1N1-32] PASSED [ 12%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-5.10.225-custom_aarch64_with_sve_and_pac-32] PASSED [ 18%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-5.10.225-custom_v1n1-32] PASSED [ 25%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-6.1.102-NO_CPU_TMPL-32] FAILED [ 31%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-6.1.102-static_V1N1-32] PASSED [ 37%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-6.1.102-custom_aarch64_with_sve_and_pac-32] PASSED [ 43%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_booted-vmlinux-6.1.102-custom_v1n1-32] PASSED [ 50%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-5.10.225-NO_CPU_TMPL-32] FAILED [ 56%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-5.10.225-static_V1N1-32] PASSED [ 62%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-5.10.225-custom_aarch64_with_sve_and_pac-32] PASSED [ 68%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-5.10.225-custom_v1n1-32] PASSED [ 75%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-6.1.102-NO_CPU_TMPL-32] FAILED [ 81%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-6.1.102-static_V1N1-32] PASSED [ 87%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-6.1.102-custom_aarch64_with_sve_and_pac-32] PASSED [ 93%]
integration_tests/functional/test_cpu_all.py::test_all_vcpus_online[uvm_restored-vmlinux-6.1.102-custom_v1n1-32] PASSED [100%]

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • [ ] I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • [ ] I have mentioned all user-facing changes in CHANGELOG.md.
  • [ ] If a specific issue led to this PR, this PR closes the issue.
  • [ ] When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • [ ] I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

Sometimes we would like to test a custom CPU template for debugging /
testing purposes. The new pytest option `--custom-cpu-template` allows
us to apply the given CPU template to any microVM in any test unless it
is overwritten by a test. Since `cpu_template_any` yields not only CPU
templates but also None, the given CPU template will be used in the None
case.

Signed-off-by: Takahiro Itazuri <[email protected]>
@zulinx86 zulinx86 requested a review from pb8o December 30, 2024 14:38
@zulinx86 zulinx86 self-assigned this Dec 30, 2024
@zulinx86 zulinx86 added the python Pull requests that update Python code label Dec 30, 2024
Copy link

codecov bot commented Dec 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.93%. Comparing base (80b4cb4) to head (991bd92).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4975   +/-   ##
=======================================
  Coverage   83.93%   83.93%           
=======================================
  Files         248      248           
  Lines       27791    27791           
=======================================
  Hits        23327    23327           
  Misses       4464     4464           
Flag Coverage Δ
5.10-c5n.metal 84.51% <ø> (ø)
5.10-m5n.metal 84.49% <ø> (?)
5.10-m6a.metal ?
5.10-m6g.metal 80.61% <ø> (ø)
5.10-m6i.metal 84.49% <ø> (ø)
5.10-m7g.metal 80.61% <ø> (ø)
6.1-c5n.metal 84.51% <ø> (ø)
6.1-m5n.metal 84.50% <ø> (ø)
6.1-m6a.metal 83.78% <ø> (ø)
6.1-m6g.metal 80.61% <ø> (ø)
6.1-m6i.metal 84.49% <ø> (ø)
6.1-m7g.metal 80.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zulinx86 zulinx86 added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Dec 30, 2024
tests/conftest.py Show resolved Hide resolved
@pb8o pb8o merged commit 5b706ed into firecracker-microvm:main Dec 30, 2024
6 of 7 checks passed
@zulinx86 zulinx86 deleted the custom_cpu_template_test_framework_2 branch December 30, 2024 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests that update Python code Status: Awaiting review Indicates that a pull request is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants