Nesterov MomentumΒΆ
- $$\begin{align}
&v_{t}= \gamma v_{t+1}+\eta \cdot \nabla_{\theta}J(\theta - \gamma v_{t-1}) \
&\theta = \theta- v_{t}\
\end{align}$$
&v_{t}= \gamma v_{t+1}+\eta \cdot \nabla_{\theta}J(\theta - \gamma v_{t-1}) \
&\theta = \theta- v_{t}\
\end{align}$$