Effect of Depth

  • Adding skip connections make the loss surface smoother

Deeper Architectures

  • Makes more uneven and chaotic

Wider Architectures

  • Makes landscape smoother and flatter