Accelerate training of restricted Boltzmann machines via iterative conditional maximum likelihood estimation

Restricted Boltzmann machines (RBMs) have become a popular tool of feature coding or extraction for unsupervised learning in recent years. However, there still lacks an efficient algorithm for training the RBM due to that its likelihood function contains an intractable normalizing constant. The existing algorithms, such as contrastive divergence and its variants, approximate the gradient of the likelihood function using Markov chain Monte Carlo. However, the approximation is time consuming and, moreover, the approximation error often impedes the convergence of the training algorithm. This paper proposes a fast algorithm for training RBMs by treating the hidden states as missing data and then estimating the parameters of the RBM via an iterative conditional maximum likelihood estimation approach, which avoids the issue of intractable normalizing constants. The numerical results indicate that the proposed algorithm can provide a drastic improvement over the contrastive divergence algorithm in RBM training. This paper also presents an extension of the proposed algorithm for how to cope with missing data in RBM training and illustrates its application using an example about drug-target interaction prediction.

Keywords

collaborative filtering, imputation-regularized optimization algorithm, missing data, stochastic EM algorithm

2010 Mathematics Subject Classification

Primary 62G99. Secondary 62P99.

Full Text (PDF format)

Liang’s research was supported in part by the grants DMS-1612924, DMS/NIGMS R01-GM117597, and NIGMS R01-126089.

Received 17 May 2018

Published 4 June 2019