Embedding with 0 masking

[ktamiola]

Just an idea! Instead of weights.

From: https://keras.io/layers/embeddings/

mask_zero: Whether or not the input value 0 is a special "padding" value that should be masked out. This is useful when using recurrent layers which may take variable length input. If this is True then all subsequent layers in the model need to support masking or an exception will be raised. If mask_zero is set to True, as a consequence, index 0 cannot be used in the vocabulary (input_dim should equal size of vocabulary + 1).

[jandom]

We can used it to mask out the padding and hopefully also the missing data

[ktamiola]

@jandom exactly. Weights were an awesome remedy, but as you might have noticed, networks still were trying to predict points at the boundaries, which is a bit worrisome.