MILAN

• Natural Language Descriptions of Deep Visual Features
• Some neurons in deep networks specialize in recognizing highly specific perceptual, structural, or semantic Features of inputs
• identifying neurons that respond to individual concept categories
• richer characterization of neuron-level computation
• mutual-information-guided linguistic annotation of neurons
• generate open-ended, compositional, natural language descriptions of individual neurons in deep networks
• generates a description by searching for a natural language string that maximizes pointwise mutual information with the image regions in which the neuron is active
• MILANNOTATIONS
• fine-grained descriptions that capture categorical, relational, and logical structure in learned Features
• characterizing the distribution and importance of neurons selective for attribute, category, and relational information in vision models.
• auditing, surfacing neurons sensitive to protected categories like race and gender in models trained on datasets intended to obscure these Features
• editing, improving robustness in an image classifier by deleting neurons sensitive to text Features spuriously correlated with class labels