Local NNs
Locally Connected Neural Networks
Inspired by the lateral inhibition in biological neuron networks, we
propose a novel neural network architecture, namely locally connected deep neural
network (LCNN), which imposes local connections within the neighboring neurons
in the same layer in order to imitate the lateral inhibition.
- Our maxout neuron based LCNN can achieve a 0.79% classfi cation error rate on the MNIST task without using pretraining and data augmentation.
- Our ReLU based LCNN can achieve a 15.0% word error rate in speech recognition on the Switchboard database.
To our knowledge, both results set the new state of the art performance on these tasks under similar training conditions.
Reference:
[1] Shiliang Zhang, Hui Jiang, Li-Rong Dai and Si Wei, ``Locally Connected Deep Neural Networks,'' submitted to Neural Networks, 2015.