Abstract
Deep neural networks have been very successful in solving hard computation tasks. Besides their problem-solving capability as an artificial intelligence approach, they have also been shown to be potential computational models of the biological neural process. However, previous comparative studies did not focus on aspects like nonlinear computations and neural variability. In a series of studies, I seek to bridge the gap between neural networks and the brain from these perspectives that are important but have been largely ignored. I searched for well-documented surround effects in standard convolutional neural networks and found that deep networks exhibited a key signature of surround effects in the early visual cortex, highlighting center stimuli that visually stand out from the surround. Further, I developed a gradient-based approach to visualize surround effects in deep neural networks. Vivid visualization revealed both documented and undocumented surround effects, which can be used as a new paradigm to study cortical surround effects. On the other hand, I explicitly incorporated brain-motivated divisive normalization computations into convolutional neural networks, and found improved image classification accuracy, especially for more shallow networks. I further developed an approach to decompose the spatial key-query interaction in vision transformer models to address a controversial problem: does self-attention attend to similar or dissimilar contexts? I analyzed and visualized the main modes of the high-dimensional key-query interaction, and found that the decomposed modes have meaningful semantic properties, including interesting conditional relationships between the object and the surrounding context. Lastly, I studied the role of dropout, which is a popular neural network building block that induces variable activations, and found structural and functional similarity to cortical neural variability.