Backend PyTorch: Add L1 and L1+L2 regularizers #1905
Conversation
|
We are unifying the regularization for tensorflow and paddle, see #1894 . Do you think we can also unify pytorch regularization in a more unified code? |
|
Unfortunately, this can be a bit difficult in pytorch. The implementation options seem unnecessarily complicated to me. |
|
I didn't notice earlier that you implemented the NysNewtonCG optimizer. Should I add L1 regularization in |
Not this PR. |
|
Sorry, it has been a while for this PR. Could you remind me what is the purpose of this PR? |
This is a pytorch implementation of L1 regularization. |
deepxde/model.py
Outdated
| ) | ||
|
|
||
| def train_step(inputs, targets, auxiliary_vars): | ||
| def closure(): | ||
| losses = outputs_losses_train(inputs, targets, auxiliary_vars)[1] | ||
| total_loss = torch.sum(losses) | ||
| if l1_factor: |
There was a problem hiding this comment.
The loss should be computed in outputs_losses, so it will be recorded and output.
There was a problem hiding this comment.
Please check it now.
Continuation of the PR #1884