-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add workflow/docs to validate changes in MESSAGE/MACRO core #180
Comments
So for a particular point of discussion, @gidden suggested:
and I said:
The reasons for this:
Hence my estimate of the time to gather the necessary info and implement these. I previously tried something similar (and spent similar time) with iiasa/message_data#411, but people did not have bandwidth to validate and review, so it was not merged. Again, I agree it's important we move in this direction, but I think we shouldn't underestimate the work required, and until there's a concrete decision to prioritize that work we should choose pragmatic alternatives. |
This expands on comments by @glatterf42 here and @gidden here. We update the description if there are alternate proposals or to link to PRs.
Summary
message-ix-models
and various branches ofmessage_data
) depend on the particular GAMS implementation of MESSAGE (and/or MACRO) in themessage_ix
package.message_ix
that modify the GAMS code, for instance Correct MACRO GDP reporting & update docs message_ix#430, Adjust calculation of PRICE_EMISSION message_ix#726, and others. Often these are to correct known bugs.message_ix
contains:message-ix-models
has a test suite with high coverage.message_data
branchdev
has low test coverage, and other branches even lower.message_ix
GAMS PRs, there is often a concern that the following could happen:message_ix
test suite, including expanded/added tests for the specific changes in a PR, passes, butmessage-ix-models
ormessage_data
tests are broken, ormessage_data
ormessage-ix-model
behaviour (the "particular outputs" mentioned in the first bullet) changes in a way that's not obvious.Things to do
There are a variety of things to do about this.
Manual checks
This is more or less what we have done thus far.
message_ix
PR author(s) or reviewer(s) say: I think this PR may have consequences "further up the stack" / "downstream", and describe what those impacts could be.Practices, e.g.
pip freeze
pip freeze
. This is currently documented under one particular "Known issue" in themessage_data
Install instructions, here, but we could maybe move this to a more prominent location, like the “Reproducibility” page of themessage-ix-models
docs.message_data
/message-ix-models
is known to work with specific versions of other packages in the stack (message-ix-models
,message_ix
,ixmp
,genno
,pandas
, any others) andmessage_data
/message-ix-models
branch/workflow; ormain
, not yet merged or associated with a PR; orrequirements.txt
file(s) should be created for that workflow/branch, recording the version(s) of upstream packages that are known to work; andSemi-automated checks
message_ix
PR author(s)/reviewer(s) must still identify: This change may impact 1 or more downstream workflow(s), in [specific ways].message_ix
(GAMS) code from the PR branch under review—rather thanmain
or the released version.The text was updated successfully, but these errors were encountered: