Dot Product Attention

• Luong et al., 2015
• $f_{att}(h_i,s_j)=h_i^T{s}_j$
• Equivalent to Multiplicative Attention with no trainable weight matrix. Performs better at larger dimensions
• Identity matrix
• $h$ is hidden state for encoder and $s$ is hidden state for decoder
• A type of Attention Alignment
• Final scores after Softmax
•