Skip to content
This repository has been archived by the owner on Apr 10, 2024. It is now read-only.

Constrained Paramaterizations #13

Merged
merged 3 commits into from
Feb 2, 2018
Merged

Constrained Paramaterizations #13

merged 3 commits into from
Feb 2, 2018

Conversation

colah
Copy link
Contributor

@colah colah commented Feb 2, 2018

Now that we have reasonable gradient overrides, we can do constrained parameterizations and constrained optimization.

I think there's two primary use cases for us:

  • Constraining optimization to epsilon balls -- this allows one to create adversarial examples.
  • Clipping the color range instead of using sigmoids. This would make it more convenient to set starting points, etc.

This commit introduces code to do both of these. The color clipping is a non-default option (and doesn't work very well at the moment) while the epsilon balls haven't been tested on whether they can make strong adversarial example attacks.

@ludwigschubert
Copy link
Contributor

Yay! \o/ I will write some smoke tests today before I suggest merging, OK? :-)

(I'm primarily aiming to have tests that at least touch most of the code once so we can spot simple incompatibility problems. It's less about proving correctness, though that'd be an added bonus.)

(projected if necessary) preconditioned gradient.
"""
inp = op.inputs[0]
inp_norm = tf.norm(inp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you using inp_norm for anything? My understanding is that the next line computes in which dimensions the inp variable is already at or beyond the edges of the unit ball, in which case you never use the actual maximum/actual norm, do you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Ludwig Schubert added 2 commits February 2, 2018 12:44

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…edup
@ludwigschubert ludwigschubert merged commit 019b49f into master Feb 2, 2018
@ludwigschubert ludwigschubert deleted the constrained-param branch February 2, 2018 22:37
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants