Relation between scaling weights of paper and implementation

[francois-rozet]

Hello @richzhang,

In the LPIPS paper, the 1x1 scaling convolution of the difference of the activations is performed before the squaring.

But, in the implementation, the difference of the activations is first squared, and after scaled.

diffs[kk] = (feats0[kk]-feats1[kk])**2
...
self.lin[kk](diffs[kk])

Is this a mistake ? If yes, in the paper or in the implementation ?

[richzhang]

Yes, the weights in the implementation correspond to w^2 in the paper. Fig 10 is also plotting w^2

[francois-rozet]

Thanks @richzhang ! Therefore, if I am not mistaken, the convolution could even be performed after the averaging.

Like,

or even

[richzhang]

You cannot collapse the channel direction before multiplying by w (which is scaling each channel).

In other words, any of these are fine:
(w y - w yhat)^2
= (w (y-yhat))^2
= w^2 (y -yhat)^2

Hope that makes sense

[francois-rozet]

Ah yes sure, my notation isn't very accurate. The norm and MSE should be spatial only.

[richzhang]

Great, yes that seems correct then! (also add a sum over channel direction, in front of w_l)

[francois-rozet]

Great, yes that seems correct then! (also add a sum over channel direction, in front of w_l)

Oh my bad, I wanted to write a dot product and not an element wise product.

In fact, it means that we can drop the nn.Conv2d for a nn.Linear, with the same weights. It might also make the code a bit faster since the product is performed on only one channel vector.

[francois-rozet]

Thank you, you can close the issue 👍