Exploding Gradient

• weight matrices have max eigenvalue $ma{x}_i{\lambda }_j>1$ , gradient increases per layer
  • if enough depth, then converges to infinity
• 
•