Inspired by the lateral inhibition in biological neuron networks, we propose a novel neural network architecture, namely locally connected deep neural network (LCNN), which imposes local connections within the neighboring neurons in the same layer in order to imitate the lateral inhibition.

• Our maxout neuron based LCNN can achieve a 0.79% classfication error rate on the MNIST task without using pretraining and data augmentation.
• Our ReLU based LCNN can achieve a 15.0% word error rate in speech recognition on the Switchboard database.

To our knowledge, both results set the new state of the art performance on these tasks under similar training conditions.

Reference:
[1] Shiliang Zhang, Hui Jiang, Li-Rong Dai and
Si Wei, ``Locally Connected Deep Neural Networks,'' submitted to Neural Networks, 2015.