Skip to content

Latest commit

 

History

History
79 lines (54 loc) · 3.67 KB

testing.md

File metadata and controls

79 lines (54 loc) · 3.67 KB

Testing With Bazel

Bazel ships with support for executing tests and generating test coverage.

Execute Tests

In general tests are executed by calling bazel test on one or multiple targets, where the targets are defined by a test rule.

  • Execute one test: bazel test //lte/gateway/python/magma/mobilityd/tests:test_uplink_gw

  • Execute all tests in folder: bazel test //lte/gateway/python/magma/mobilityd/tests:all Note: "all" is a keyword that expands all found test targets.

  • Execute all tests in lte: bazel test //lte/...:all

  • Execute all tests: bazel test //...:all

  • Why is my test not executed?

    • A reason could be a test tagged as "manual", see below.

  • Bazel not only caches build targets that are needed for tests, but also test results. This is, if there is no change in test dependencies or test logic, then a test re-run is completely fetched from the caches.

Helpful Flags

For a detailed documentation see the bazel command line reference.

Flag Effect
--test_output=all print all test output
--sandbox_debug do not delete sandbox after test
--cache_test_results=no force rerun of tests, i.e., do not use cached results

Execute Coverage

In general coverage is generated by calling bazel coverage on one or multiple targets, where the targets are defined by a test rule. See "Execute Tests" for target expanding.

Coverage data can be found in $MAGMA_ROOT/bazel-out/_coverage/_coverage_report.dat. For a quick preview you can generate a html report by, e.g.: genhtml -o genhtml bazel-out/_coverage/_coverage_report.dat The report can be opened via $MAGMA_ROOT/genhtml/index.html.

Tags

Test targets can be enriched by tags for filtering. Tags are organized in test_constants.bzl. For details see documentation in test_constants.bzl. Newly introduced tags should be added there. Tags should only be introduced when necessary so that the base set does not become cluttered.

py_test(
  ...
  tags = TAG_FOO + TAG_BAR,
)

Tags can be used to filter tests to be executed: bazel test --test_tag_filters=foo,-bar ... Execute all tests that are tagged with "foo" but exclude all that are tagged with "bar".

Tag Usage In Magma

  • Tags are not used for specifying components or modules, e.g., "lte" or "mobilityd". In order to filter on those criteria use the target expansion on folders, e.g.,
    • bazel test //lte/gateway/python/...:all all python lte tests
    • bazel test //lte/gateway/python/magma/mobilityd/...:all all python mobilityd tests
  • Unit test is the default test type, i.e., there is no tag required for these tests.
  • Non-unit-tests should be tagged by the specific test type, e.g., integration_test, load_test, ... . These tests should usually also be tagged as "manual" (see below).
    • Example: All "sudo tests" (Python specific tests that need to be executed as root) are tagged with "sudo_test" and "manual". This is handled via test_constants.bzl.

Tags With Special Semantics

Some tags have special semantics in bazel. See section "Attributes common to all build rules", "tags" for general documentation.

Currently only "manual" is used. Tests are not expanded by the "all" statement, i.e., these test targets need to be called explicitly. In order to execute multiple targets that are tagged with "manual", the query statement can be used:

bazel test --test_tag_filters=sudo_test $(bazel query "attr(tags, manual, //lte/gateway/python/magma/mobilityd/tests/...)")

Executes all "sudo_test"s in mobilityd.