build_renewable_profile_offwind-dc broken for manually built europe-2013-era5 cutout
#434
Comments
|
Does the issue only occur with What is the spatial extent of the cutout you obtain using Potentially related note: In #321 we build the cutouts using the config.yaml file from the PR (different from |
|
And: You mention the repository was freshly cloned. Just double checking that the repository was *completly fresh? You did not run any parts of the workflow prior in this folder? If you have outputs from That's a very common problem... |
|
Thanks for following up @euronion.
It doesn't work for
Yes, I cloned into a completely new directory called git clone [email protected]:pypsa/pypsa-eur.git pypsa-broBut I did not rebuild a new conda environment, I used my existing conda environment from my "main" `pypsa-eur" clone. |
❯ python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33)
[Clang 13.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import atlite
>>> cutout = atlite.Cutout(path='cutouts/europe-2013-era5.nc')
>>> cutout.bounds
array([-12.15, 32.85, 34.95, 72.15]) |
|
@euronion I think you're right on the money with that question. The retrieved and built cutouts have different extents: >>> cutout_built = atlite.Cutout(path='cutouts/europe-2013-era5.nc')
>>> cutout_retrieved = atlite.Cutout(path='../pypsa-eur/cutouts/europe-2013-era5.nc')
>>> cutout_retrieved
<Cutout "europe-2013-era5">
x = -16.50 ⟷ 40.50, dx = 0.30
y = 32.70 ⟷ 75.00, dy = 0.30
time = 2013-01-01 ⟷ 2013-12-31, dt = H
module = era5
prepared_features = ['height', 'wind', 'influx', 'temperature', 'runoff']
>>> cutout_built
<Cutout "europe-2013-era5">
x = -12.00 ⟷ 34.80, dx = 0.30
y = 33.00 ⟷ 72.00, dy = 0.30
time = 2013-01-01 ⟷ 2013-12-31, dt = H
module = era5
prepared_features = ['height', 'wind', 'influx', 'temperature', 'runoff']Do you know why that might be, and what causes |
|
In Lines 344 to 353 in 6d13b40 My guess would be that one of them has the incorrect shape leading to the error when Could you try and remove Lines 166 to 167 in 6d13b40 If that still raises the error, try to also remove |
|
I noticed that #321 removed the bounds parameters, which results in <Cutout "europe-2013-era5">
x = -16.50 ⟷ 36.90, dx = 0.30
y = 32.70 ⟷ 75.00, dy = 0.30
time = 2013-01-01 ⟷ 2013-12-31, dt = H
module = era5
prepared_features = ['height', 'wind', 'influx', 'temperature', 'runoff']And more importantly - |
|
For reference, here's a map of the 3 different bounds sets.
|
The #321 branch was a quick-and-dirty project where we also tried new things like removing the bounds. We didn't adapt all the changes back to the
You mean after removing the |
After removing the bounds. I didn't try removing |
|
Interesting. I still don't understand what is the root of the issue. Since you have two pypsa-eur folders at hand, could you compare the potentials for both They should not be different. |
|
Update: removing the |
|
Hey you two, this seems really hard to track down. Removing the bounds in the config works without any known problems, so we could remove these. But it would not be a satisfying fix. It seems it has to do with the most nothern areas and the |
|
I guess it has something to do with the fact that the raster data from the shipping density and the actual cutout raster do not overlap. The shipping density raster is only from x: -16.8 to 40.6 and y: 75.3 to 32.4 and therefore a bit smaller along the x-axis. Normally, Atlite should be able to fill the missing area if the argument |
|
@p-glaum for me the shipping raster is larger than the cutout. (both downloaded from zenodo). Do we use the same data? The error is still raised. |
|
@euronion I was using my own cutout. So in your case, you were using the cutout and shipping raster from zenodo and it wasn't working? |
|
No, I was comparing the cutout bounds with the shipping raster bounds (both from zenodo and build locally). In both cases the cutout is within the shipping raster. Turns out the issue is not created by the rasters, but actually be the
If I remove these region shapes, then all scripts run without an issue.
Your thoughts @FabianHofmann @p-glaum ? |
|
Hi, Just to put my two cents in, I found the dependency of the cutouts on the network Adding in years and other parameters as wildcards to the snakemake workflow, anything which requires to build whole copies of everything from The latest version now has support for new projects with an optional shared cutout folder, almost exactly what I've modified my workflow to, but it sacrifices all the snakemake functionality in the process. It's something that now has to be tracked manually. I'm not sure what I think is the best way forward, but maybe if the spatial and temporal features of |
Checklist
masterbranch or the latest release. Please indicate.pypsa-eurenvironment. Update viaconda env update -f envs/environment.yaml.Describe the Bug
Attempting to build renewable profile for off-shore wind DC with
build_cutout: truein a freshly clonedpypsa-eurrepo fails withValueError: operands could not be broadcast together with shapes (247,221) (57,194)My configuration changes are shown below - I removed the
sarahcutout, replaced its usages with theera5cutout, and setbuild_cutout: true.Configuration Changes
Error Message
The error occurs on the line
exclusions = exclusions | masked_ingis.pyinatlite.As far as I understand, the error message indicates that I'm attempting a matrix operation on matrices with incompatible sizes (presumably
exclusionsandmaskedhave different sizes), but I don't understand why that happens - and why it works with a retrievedeurope-2013-era5cutout.Full Error Message
The text was updated successfully, but these errors were encountered: