Matrix notation for NNs

Example for simple case

• 
• A NN consists of linear transformations alternating with loss functions.
• 
• or , \begin{align*} \mathbf{h = a[\theta_{0}+ \theta_{x}]} \\ \mathbf{h' = a[\psi_{0} + \Psi h]}\\ \mathbf{y' = \phi'_{0}+ \phi'h'} \end{align*}
  • where $\mathbf{a}[\bullet ]$

General case

• Vector of hidden units at layer $k$ is $h_k$
• Vector of biases that contribute to hidden layer $k+1$ is ${\beta }_k$
• Weights that are applied to $k^{th}$ layer and contribute to hidden layer $(k+1{)}^{th}$ is ${\Omega }_k$
• General network $y=f[x,\varphi ]$ with $K$ layers is now
•