How to ignore some pixels when calculating the loss?

[MargauxMasson]

Hi,

I would like to ignore some pixels in the pairs of images I am comparing, how can I do this? I have the masks of the pixels I don't want to be used when calculating the loss.

Thank you!

[richzhang]

You can turn spatial=True https://github.com/richzhang/PerceptualSimilarity/blob/master/lpips/lpips.py#L23 and then multiply by the mask. Keep in mind that this won't mask the pixels out exactly, since the deep features have a receptive field. To truly mask out the pixels, you need to trace through the receptive fields of the deep activations like in https://openaccess.thecvf.com/content_cvpr_2017/papers/Gatys_Controlling_Perceptual_Factors_CVPR_2017_paper.pdf