Using Perceptual Similarity as a loss function for a Neural Network

[harbarex]

I am trying to replace the standard loss functions like MSE in my autoencoder network, with the Perceptual Similarity Metric. I wanted to know whether this would be possible, since instances of the network and specific formatting might be required for the same.

[richzhang]

I didn't understand the 2nd half of the sentence. The 1st half should be doable.

[harbarex]

I didn't understand the 2nd half of the sentence. The 1st half should be doable.

I was taking the reference of this issue: keras-team/keras#10529
Since the function needs to be computed at every epoch, would any change be required for the present code you have provided?

I had modified the test script to work for images, but am unsure how it would fit into a neural network

[richzhang]

Simply call the forward function, whenever you need the loss computed

[harbarex]

Hi Mr. Zhang,
I am still trying to get the similarity working as a loss function, and am getting the following error:

ValueError: Dimensions must be equal, but are 224 and 3 for '{{node computeSimilarity/sub}} = Sub[T=DT_FLOAT](computeSimilarity/unstack, computeSimilarity/sub/y)' with input shapes: [224,224,3], [1,3,1,1].

I am using images of dimensions 224 x 224 x 3, and a batchSize of 8. The following is the custom loss function I am using for my code:

def computeSimilarity(image1, image2):

	'''
	im1 = im2tensor(image1)
	im2 = im2tensor(image2)

	loss = loss_fn_alex(im1, im2)
	'''

	batchSize = image1.shape[0]

	im1 = tf.unstack(image1)
	im2 = tf.unstack(image2)

	loss = 0

	for i1,i2 in zip(im1,im2):
		loss += loss_fn_alex(i1, i2)

	loss = loss / batchSize

	return float(loss)

Kindly let me know what the issue is

[richzhang]

Make sure the images are in NxCxHxW form (this is the standard way to represent images in pytorch). You can refer to the demo script as well: https://github.com/richzhang/PerceptualSimilarity/blob/master/lpips_2imgs.py