Berkeley Et Al

• 576 input patterns
• 5793 epochs needed to reach convergence
• Model frozen, stimulus set presented again
• Activation of each hidden unit recorded
• Single unit recording
• Why do these bands appear?
• Gaussian activation function
• But banding patterns have been found with sigmoidal activation as well
• Effect = units only respond to a limited number of inputs
• Bands appear when weights into HUs cancel each other out
• Activations of hidden neurons can be organized into bands
• Bands are associated with interpretable Features
• Lesion studies show bands are essential to solving problem
• For some problems under some circumstances, neural networks develop highly selective hidden units
• Looks like localist coding (grandmother cells)
• Patterns of activation can be ambiguous on their own
• But realistically, more than one pattern might be activated simultaneously
• Superimposing two or more patterns over same units leads to an ambiguous blend
• Problem for localist representations, but even more serious with distributed representations