ci: cache conda envs betweeen successive builds in a day #3525
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3525 +/- ##
==========================================
+ Coverage 82.76% 82.82% +0.06%
==========================================
Files 199 199
Lines 27339 27360 +21
==========================================
+ Hits 22627 22661 +34
+ Misses 4712 4699 -13
Continue to review full report at Codecov.
|
| include: | ||
| - os: ubuntu-latest | ||
| path: /usr/share/miniconda3/envs/ | ||
| - os: macos-latest | ||
| path: /Users/runner/miniconda3/envs/ | ||
| - os: windows-latest | ||
| path: C:\Miniconda3\envs\ |
There was a problem hiding this comment.
I've seen that these absolute paths have been removed from the documentation of the setup-miniconda action. Maybe they're no longer required?
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ matrix.path }} | ||
| key: ${{ runner.os }}-conda-${{ matrix.python-version }}-${{ hashFiles('setup.py') }}-${{ env.TODAY }}-${{ env.CACHE_VERSION }} |
There was a problem hiding this comment.
It'd be nice to have the installed group extras added in the cache key, as done in hvplot:
We may use caching for the docs workflow for instance (we sometimes trigger many docs build in a day, that could help), and need to make sure the tests key not the same as the docs key. Maybe instead of adding the extras group it'd make more sense to add a hash of the test.yaml file itself?
| restore-keys: | | ||
| ${{ runner.os }}-conda-${{ matrix.python-version }}-${{ env.TODAY }}-${{ env.CACHE_VERSION }} |
There was a problem hiding this comment.
In hvplot we had removed that, because of the different workflow types we may run in a single day.
| @@ -99,6 +121,7 @@ jobs: | |||
| run: | | |||
| eval "$(conda shell.bash hook)" | |||
| conda activate test-environment | |||
| python -m pip install --no-deps --no-build-isolation -e . | |||
There was a problem hiding this comment.
Could you move this to its own step before the doit env_capture step? I like to have the doit env_capture step super simple, that's a least one step I know is not causing any issue when I debug the CI 🙃
|
Thanks for this, it provided a good template. We've now implemented caching as part of a composite action: https://github.com/pyviz-dev/holoviz_tasks/tree/main/install |
Quicker turn around time for github actions. The cache is invalidated every day once (date is used in the key for the cache). It will also rerun if there is any change to setup.py.
There is a similar setup for hvplot.