RuntimeError: Function 'SqrtBackward' returned nan values in its 0th output.

[niels-leif-bracher]

Hi,
i wanted to ask if its possible to add the epsilon under the sqrt in the norm function instead of adding it under the fraction because I think for my use case some sqrt are 0 and therefore the gradient returns nan.

def normalize_tensor(in_feat,eps=1e-10):
norm_factor = torch.sqrt(torch.sum(in_feat**2,dim=1,keepdim=True))
return in_feat/(norm_factor+eps)

[niels-leif-bracher]

Somehow switching from vgg to alexnet solved this issue.

[MartinPernus]

Having the same problem using alexnet. Solved by adding epsilon under the sqrt.

[Sinjini15]

Hi @MartinPernus , do you mean you added an epsilon within the sqrt calculation? I am having the same error