-
Notifications
You must be signed in to change notification settings - Fork 500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Difference Paper - Implementation #18
Comments
normalize_tensor doesn't take the norm, it normalizes the tensor. I think the implementation matches the paper. |
Hello, lets say we have the following example:
we normalize the two inputs a and b:
now from the paper formula i would expect the following computation:
in the implementation its computed like this:
which would be equivalent to shouldn't it be:
or did I miss something? Thanks in advance! |
Each element needs to be squared first (see how p-norms are calculated) |
Inside the || || is a vector, not a scalar. The symbol does not mean dot product |
Sorry, somehow missed the reply. Thanks! |
Hi, thank you for your amazing work, but I also find the code a bit different from the paper: |
Yes, the |
Thank you so much for your reply. May I ask another question? |
Hmm, the weights should match the code. Many of the weights are 0 in the official model as well (see Fig 10 in the paper: https://arxiv.org/pdf/1801.03924.pdf), depending on the variant |
Yes. Thank you. I checked the official model, most weights are close to 0 (<0.1) but not actually 0. Mine re-trained model has most weights equal to 0. I will debug more. Thanks! |
Dear authors,
equation (1) in the paper states that you are taking the euclidean norm squared of the weighted differences.
Something like
euclidean_norm(dot(w_l, (y - y_0)))²
However, in the implementation you are weighting the squared difference of the euclidean norms, something like
dot(w_l, (euclidean_norm(y)-euclidean_norm(y_0))²)
which as far as I am concerend is not the same thing. Or am I missing something here?Thanks!
The text was updated successfully, but these errors were encountered: