Noise Tunnel

• @kokhlikyanCaptumUnifiedGeneric2020

Summary

• Combines [SmoothGrad Square] + [Smooth-Grad] + [VarGrad](SmoothGrad Square] + [Smooth-Grad] + [VarGrad.md)
• not an attribution method

Using Smooth-Grad

• technique that improves the accuracy of attribution methods
• problem with ReLU activation function and gradients producing noisy, often irrelevant attributions
• the partial derivative $\frac{\partial F_c}{\partial x_i}$ of the models’ score $F_c$​ for a class c with respect to the value of the pixel $x_i$ fluctuates
• adding a Gaussian noise $\mathcal{N}(0,0.01^2)$ and calculating an average of sampled attributions is going to solve the problem
• calculates the attribution $M_c$ using any available method by providing that method an input with Gaussian noise
• calculates a mean value from all the samples to reduce the importance of less frequent attributions
• when adding noise to the input image, important attributions are going to be visible most of the time, and noise might change between attributions
• ${\hat{M}}_c(x)=\frac{1}{n}{\Sigma }_1^n{M}_c(x+\mathcal{N}(0,{\sigma }^2))$

Using [SmoothGrad Square](SmoothGrad Square.md)

• changes only the way that the mean value is calculated by using the mean of squared attributions instead of just attributions
• less noisy results
• but often removes less important features, which are still valid features
• ${\hat{M}}_c(x)=\frac{1}{n}{\Sigma }_1^n\sqrt{M_c(x+\mathcal{N}(0,{\sigma }^2))}$

Using VarGrad

• variance version of the SmoothGrad
• Using SmoothGrad (Noise Tunnel) seems to detect more edges of the input image (in comparison with pure IG attribution in [Fig. 1b]), and that can be interpreted as detecting decision boundary. SmoothGrad-Square (Noise Tunnel) and VarGrad (Noise Tunnel) are removing a large amount of noise but usually also some of the important features visible on the attribution from SmoothGrad
• ${\hat{M}}_c(x)=\frac{1}{n}{\Sigma }_{k=1}^n\{M_c(x+\mathcal{N}(0,{\sigma }^2)){\}}^2-\{{\hat{M}}_c(x){\}}^2$

Drawbacks

• Even if the Noise Tunnel method improves the accuracy of the XAI methods it adds a large amount of computational overhead
• Every sample generated by the method requires the rerun of the whole XAI method
• That is a linear increase of computation and to make the method efficient you should use at least 5 generated noise samples

Images

•