Block Sparse Kernel
- For networks with block sparse weights
- Can choose amount of Sparsity
- Can replace normal Dense Layers with sparse and wide or sparse and deep
- Enables wider and deeper networks
- Only compute on non zero blocks
- Connectivity is unaffected in the spatial dimensions
- Compute cost is only prop to number of non zero weights
- Small World graphs
- Also useful for compression