This paper seeks to scale up Bayesian learning of RNNs to meet the challenge of the increasing amount of "big" sequential data in natural language processing, leveraging recent advances in stochastic gradient Markov Chain Monte Carlo (SG-MCMC) algorithms (Welling and Teh, 2011;Ding et al., 2014;Li et al., 2016a,b).
instead of training a single network, SG-MCMC is employed to train an ensemble of networks, where each network has its parameters drawn from a shared posterior distribution.