Tests fail with absl.flags._exceptions.UnparsedFlagAccessError

[theo-brown]

I have cloned a clean copy of the repo and set up my environment from scratch, installing a clean Python (3.13) and running pip install -e .[dev] in the torax repo.

However, currently running pytest errors with:

=========================================================================================== ERRORS ===========================================================================================
__________________________________________________________ ERROR collecting torax/tests/test_data/test_iterhybrid_rampup_restart.py __________________________________________________________
torax/tests/test_data/test_iterhybrid_rampup_restart.py:25: in <module>
    paths.test_data_dir(), 'test_iterhybrid_rampup.nc'
torax/tests/test_lib/paths.py:25: in test_data_dir
    src_dir = absltest.TEST_SRCDIR.value
../../../.conda/envs/torax/lib/python3.12/site-packages/absl/flags/_flagvalues.py:1426: in value
    val = getattr(self._flagvalues, self._name)
../../../.conda/envs/torax/lib/python3.12/site-packages/absl/flags/_flagvalues.py:498: in __getattr__
    raise _exceptions.UnparsedFlagAccessError(
**E   absl.flags._exceptions.UnparsedFlagAccessError: Trying to access flag --test_srcdir before flags were parsed.**
================================================================================== short test summary info ===================================================================================
ERROR torax/tests/test_data/test_iterhybrid_rampup_restart.py - absl.flags._exceptions.UnparsedFlagAccessError: Trying to access flag --test_srcdir before flags were parsed.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================================================== 1 error in 1.60s ======================================================================================

My environment is:

  Name              Version       Build                    Channel    
────────────────────────────────────────────────────────────────────────
  _libgcc_mutex     0.1           conda_forge              conda-forge
  _openmp_mutex     4.5           2_gnu                    conda-forge
  blosc             1.21.6        hef167b5_0               conda-forge
  bzip2             1.0.8         h4bc722e_7               conda-forge
  c-ares            1.34.1        heb4867d_0               conda-forge
  ca-certificates   2024.8.30     hbcca054_0               conda-forge
  certifi           2024.8.30     pyhd8ed1ab_0             conda-forge
  cftime            1.6.4         py313ha014f3b_1          conda-forge
  hdf4              4.2.15        h2a13503_7               conda-forge
  hdf5              1.14.4        nompi_h2d575fe_101       conda-forge
  icu               75.1          he02047a_0               conda-forge
  keyutils          1.6.1         h166bdaf_0               conda-forge
  krb5              1.21.3        h659f571_0               conda-forge
  ld_impl_linux-64  2.43          h712a8e2_1               conda-forge
  libaec            1.1.3         h59595ed_0               conda-forge
  libblas           3.9.0         24_linux64_openblas      conda-forge
  libcblas          3.9.0         24_linux64_openblas      conda-forge
  libcurl           8.10.1        hbbe4b11_0               conda-forge
  libedit           3.1.20191231  he28a2e2_2               conda-forge
  libev             4.33          hd590300_2               conda-forge
  libexpat          2.6.3         h5888daf_0               conda-forge
  libffi            3.4.2         h7f98852_5               conda-forge
  libgcc            14.1.0        h77fa898_1               conda-forge
  libgcc-ng         14.1.0        h69a702a_1               conda-forge
  libgfortran       14.1.0        h69a702a_1               conda-forge
  libgfortran-ng    14.1.0        h69a702a_1               conda-forge
  libgfortran5      14.1.0        hc5f4f2c_1               conda-forge
  libgomp           14.1.0        h77fa898_1               conda-forge
  libiconv          1.17          hd590300_2               conda-forge
  libjpeg-turbo     3.0.0         hd590300_1               conda-forge
  liblapack         3.9.0         24_linux64_openblas      conda-forge
  libmpdec          4.0.0         h4bc722e_0               conda-forge
  libnetcdf         4.9.2         nompi_h2564987_115       conda-forge
  libnghttp2        1.58.0        h47da74e_1               conda-forge
  libopenblas       0.3.27        pthreads_hac2b453_1      conda-forge
  libsqlite         3.46.1        hadc24fc_0               conda-forge
  libssh2           1.11.0        h0841786_0               conda-forge
  libstdcxx         14.1.0        hc0a3c3a_1               conda-forge
  libstdcxx-ng      14.1.0        h4852527_1               conda-forge
  libuuid           2.38.1        h0b41bf4_0               conda-forge
  libxml2           2.12.7        he7c6b58_4               conda-forge
  libzip            1.11.1        hf83b1b0_0               conda-forge
  libzlib           1.3.1         hb9d3cd8_2               conda-forge
  lz4-c             1.9.4         hcb278e6_0               conda-forge
  ncurses           6.5           he02047a_1               conda-forge
  netcdf4           1.7.1         nompi_py313h1dd084c_103  conda-forge
  numpy             2.1.2         py313h4bf6692_0          conda-forge
  openssl           3.3.2         hb9d3cd8_0               conda-forge
  pip               24.2          pyh145f28c_1             conda-forge
  python            3.13.0        h9ebbce0_100_cp313       conda-forge
  python_abi        3.13          5_cp313                  conda-forge
  readline          8.2           h8228510_1               conda-forge
  setuptools        75.1.0        pyhd8ed1ab_0             conda-forge
  snappy            1.2.1         ha2e4443_0               conda-forge
  tk                8.6.13        noxft_h4845f30_101       conda-forge
  tzdata            2024b         hc8b5060_0               conda-forge
  xz                5.2.6         h166bdaf_0               conda-forge
  zlib              1.3.1         hb9d3cd8_2               conda-forge
  zstd              1.5.6         ha6fb4c9_0               conda-forge

[jcitrin]

Thanks for flagging this (no pun intended). I can reproduce this. Looking into it.

[jcitrin]

I think the issue is the blanket pytest is discovering "tests" that should not be run., for example the config files in tests/tests_data/*.py are not actually tests.

The list of tests run with GitHub Actions are described in torax/.github/workflows/pytest.yml . See e.g. "Run core tests". For now you can try running some of those to see if they work (sim tests are recommended with sharding).

In the meantime we'll look at a solution for running the appropriate tests locally from a single command. Might be as simple as a utility script + improved documentation.

[theo-brown]

Yep, that's probably it. This wasn't stopping me from doing what I needed to do, just wanted to raise it as I caught it and it was clearly unwanted/undocumented behvaiour. The genuine tests run fine!

[jcitrin]

Has everything been resolved now? Can we close this issue?

[theo-brown]

Yes, all resolved now, thanks to #443.