Facebook AI Says Class Selectivity In Neurons May Impair DNN Performance

New Facebook AI research proposes a rethinking of the role of class selective neurons in deep neural networks (DNN), arguing DNNs can perform well — and maybe even better — without them. The researchers say the findings suggest overreliance on such easy-to-interpret neurons and intuition-based methods for understanding DNNs can be misleading.

DNNs are built with multiple layers between the input and output layers and are often used to model complex non-linear relationships. Although the machine learning community has achieved SOTA results with DNNs, understanding what these networks actually do remains a challenge. Such interpretability concerns have given rise to the “black box” stigma that continues to impede AI deployment.

The team says selectivity is currently widely used in part because it’s intuitive and easy-to-understand in human terms, and also because these kinds of interpretable neurons do naturally emerge in networks trained on a variety of tasks. For example, DNNs trained to classify many different kinds of images contain individual neurons that activate most strongly for Labrador retrievers. And DNNs trained to predict individual letters in product reviews contain neurons selective for positive or negative sentiment.

The researchers tackled the question of whether such easy-to-interpret neurons are actually necessary by training an image classification network to improve accuracy as usual, but also adding an incentive to decrease or increase the amount of class selectivity in its neurons. They controlled the importance of class selectivity to the network using a single parameter that chooses between encouraging or discouraging the interpretable neurons and to what degree.

To their surprise, reducing DNNs’ class selectivity had little adverse effect and in some cases even improved performance. Increasing class…

Related posts:

My mother used to read with me on the sofa every night. We had no air conditioning, just screens on the large windows on three sides of the house to let the trade winds pass through. We never had a…

Everything in my life. “Scratch Paper” is published by mia laurén herrera.