dimension is not compatible with "net" model.

[lamourj]

When using --model net for using an off-the-shelf network, train.py breaks here:

PerceptualSimilarity/models/networks_basic.py

Line 194 in 2416334

return self.model.forward(torch.cat((d0,d1,d0-d1,d0/(d1+eps),d1/(d0+eps)),dim=1))

because the dimensionality don't match. When using net-lin models, the dimensionality is [50, 1, 1, 1] while this wrapping into arrays doesn't happen when using net models.
Is there a reason for this wrapping under the net-lin model?

[lamourj]

Here is how a tensor looks at this point in the code when using the net-lin model:

tensor([[[[-2.6783e-04]]], [[[-1.1941e-03]]], [[[-2.8335e-03]]], [[[-8.6104e-03]]], [[[-4.1192e-05]]], [[[-8.8206e-04]]], [[[-2.5866e-04]]], [[[-2.8071e-05]]], [[[-6.7918e-05]]], [[[-7.5164e-03]]], [[[-5.7435e-05]]], [[[-1.9981e-03]]], [[[-5.1527e-05]]], [[[-8.1360e-05]]], [[[-6.0457e-05]]], [[[-1.9554e-03]]], [[[-1.4526e-03]]], [[[-1.5009e-03]]], [[[-5.9766e-03]]], [[[-3.6634e-05]]], [[[-2.4013e-05]]], [[[-1.5150e-04]]], [[[-1.4569e-03]]], [[[-1.1102e-03]]], [[[-5.2675e-04]]], [[[-9.3123e-05]]], [[[-2.6657e-04]]], [[[-1.8474e-04]]], [[[-2.2805e-03]]], [[[-4.1619e-04]]], [[[-1.9141e-04]]], [[[-8.8367e-04]]], [[[-1.8202e-04]]], [[[-2.8138e-04]]], [[[-3.3072e-04]]], [[[-3.2947e-04]]], [[[ 3.1789e-06]]], [[[-2.4202e-03]]], [[[-1.8145e-04]]], [[[-1.0697e-03]]], [[[ 6.1007e-06]]], [[[-1.6801e-04]]], [[[-2.4768e-03]]], [[[-4.1269e-04]]], [[[-2.1366e-03]]], [[[-1.8986e-05]]], [[[-1.1094e-04]]], [[[-1.4528e-03]]], [[[-1.0085e-03]]], [[[-7.3988e-04]]]], grad_fn=<ViewBackward>)

[richzhang]

The flag --model net is for using an off-the-shelf network, weighting all of the channels the same and is not meant for training. If you want to train a linear layer on top, do --model net-lin. Thanks!

[lamourj]

Thank you for your answer @richzhang ! Out of curiosity, is there any way to train the CNN weights instead of only the linear layer on top with your code ? Thanks!

[richzhang]

Yes. Use --train_trunk flag. Additionally, you can use the --from_scratch flag to train from scratch. Example commands finetuning the trunk from pretrained weights and training the trunk all the way from scratch.

[lamourj]

Ok got it! That's what I thought, great to have the confirmation. Thanks again!