Skip to content

Commit

Permalink
Use tf.softmax_cross_entropy_with_logits to calculate loss (tensorflo…
Browse files Browse the repository at this point in the history
…w#181)

* Use the tensorflow cross entropy function to prevent nan losses.

* Correct double softmax and use mean for loss.
  • Loading branch information
Evan Lezar authored and martinwicke committed Jun 23, 2016
1 parent 05630a7 commit d816971
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 4 deletions.
2 changes: 1 addition & 1 deletion transformer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ A Spatial Transformer Network implemented in Tensorflow 0.7 and based on [2].
</div>

```python
transformer(U, theta, downsample_factor=1)
transformer(U, theta, out_size)
```

#### Parameters
Expand Down
7 changes: 4 additions & 3 deletions transformer/cluttered_mnist.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,16 +119,17 @@
# %% And finally our softmax layer:
W_fc2 = weight_variable([n_fc, 10])
b_fc2 = bias_variable([10])
y_pred = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
y_logits = tf.matmul(h_fc1_drop, W_fc2) + b_fc2

# %% Define loss/eval/training functions
cross_entropy = -tf.reduce_sum(y * tf.log(y_pred))
cross_entropy = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(y_logits, y))
opt = tf.train.AdamOptimizer()
optimizer = opt.minimize(cross_entropy)
grads = opt.compute_gradients(cross_entropy, [b_fc_loc2])

# %% Monitor accuracy
correct_prediction = tf.equal(tf.argmax(y_pred, 1), tf.argmax(y, 1))
correct_prediction = tf.equal(tf.argmax(y_logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, 'float'))

# %% We now create a new session to actually perform the initialization the
Expand Down

0 comments on commit d816971

Please sign in to comment.