Soft Attention

• For a simple Seq2Seq, all hidden state vectors $h_t$ across timesteps are linearly combined
• $c_i={\Sigma }_{j=1}^T{\alpha }_{ij}{h}_j$
• $a_{ij}=\frac{exp(e_{ij})}{{\Sigma }_{k=1}^{T}exp(e_{ij})}$
•