AttentionCell

Summary: This diff creates a generalized AttentionCell class, which will allow us to construct attention decoders out of arbitrary RNNCell components (with a particular view to using stacked, multi-layer RNNs). In order to do this, we introduce a new optional input for RNNCell._apply which allows us to provide an additional input that is not processed by prepare_input(). Note that this is an argument only to _apply, not apply, since it is only meant to be used for additional recurrent connections to "embedded" cells, not for standalone RNNs. Reviewed By: urikz Differential Revision: D4998465 fbshipit-source-id: 473009ea4917e86e365f9d23aa2f11a46a94fd65
aaaqhbd · May 5, 2017 · 8530ac3 · 8530ac3
1 parent 8be1311
commit 8530ac3
Show file tree

Hide file tree

Showing 2 changed files with 266 additions and 305 deletions.
diff --git a/caffe2/python/operator_test/rnn_cell_test.py b/caffe2/python/operator_test/rnn_cell_test.py
@@ -848,5 +848,5 @@ def test_multi_lstm(
                 input_to_check=str(param),
                 print_net=False,
                 step_size=0.0001,
-                threshold=0.05,
+                threshold=0.1,
             )