Speaker
Mr
Ilsang Ohn
(Department of Statistics, Seoul National University)
Description
Sparsity is a key ingredient in the success of learners both theoretically and computationally. This is also the case for deep neural networks (DNNs). A number of empirical observations show that sparse DNNs can dramatically reduce computation time and memory without appreciably harming prediction power. Furthermore, recent theoretical studies proved that DNN estimators with a certain sparsity constraint can attain optimal convergence rates for regression and classification problems. However, they only considered the empirical risk minimizer under the sparsity constraint, where optimization is almost impossible in practice due to its discrete nature. In this research, we propose a novel penalized empirical risk minimization method for estimating sparse DNNs with a scalable computation algorithm. The proposed method yields sparse DNNs that can achieve optimal convergence rates of excess risks for various learning problems including regression and binary classification. We demonstrate the empirical performance of the proposed method and compare it with other competitors for various benchmark datasets.
Primary author
Mr
Ilsang Ohn
(Department of Statistics, Seoul National University)
Co-author
Prof.
Yongdai Kim
(Department of Statistics, Seoul National University)
