Communications in Mathematical Sciences
Volume 19 (2021)
Regularized least square kernel regression for streaming data
Pages: 1533 – 1548
We study the use of kernel ridge regression (KRR) in the block-wise streaming data. The algorithm works in an online manner: when a new data block comes in, the algorithm computes a local estimator based on the incoming data block and updates the predictive model by weighted average of all local estimators. Assuming the block data sizes increase at a mild rate and the regularization parameters are selected adaptively according to the sample size of all available data at the time of updating the model, we prove the convergence of the average KRR estimator. The rate is optimal when the regression function can be well approximated by the reproducing kernel Hilbert space in the $L^2$ sense.
learning theory, kernel ridge regression, streaming data, online learning, adaptive underregularization
2010 Mathematics Subject Classification
68Q32, 68T05, 68W27
The work by Hongwei Sun described in this paper is supported by National Natural Science Foundation of China (Grants No. 11671171 and 11871167).
The work by Qiang Wu is partially supported by the Simons Foundation Collaboration Grant (Award ID 712916).
The three authors contributed equally to this paper.
Received 14 July 2020
Accepted 27 January 2021
Published 2 August 2021