Citation Author(s):
Fei Wang
Submitted by:
Fei Wang
Last updated:
Thu, 11/08/2018 - 10:34
Data Format:
Dataset Views:
Share / Embed Cite

This is the code and data for IS-RNN validation and research


# Importance Sampling for RNNs


1. Prerequisite


a. Install Theano

b. Nvidia GPU with memory > 8G

c. Install CUDA 9.0 and Cudnn > 7.0

d. Install Libgpuarray for theano

e. Install midi package for python (not in pip repo), we include this package in 'code' folder.


2. Run


    a. ./ in 'code' folder for IS-RNN, IS-LSTM and IS-RNN-RBM.

    b. ./ in 'nmt_code' folder for IS-GRU-NMT.


3. Visualization


    a. ./, ./ and ./ in 'code' folder

    b. ./ in 'nmt_code' folder


4. Misc


    a. Change interval which controls the IS interval as you wish, we empirically set 

       it as 3 or 4. In fact, it can be larger and IS still achieves accelerated convergence rate.


    b. We collect gradient norm and loss, score, etc at the end of every epoch for results presentation.

       Be noted that the gradient norm calculated is only used to generate new IS sequence when epoch%interval==0.

       In other epochs, the gradient norm is only calculated for results presentation.

       Comment out this section and thus the gradient norm is actually calculated every interval epochs.


    c. Update and optimization can be find on


    d. For IS-LSTM case, you need GPU memory >12G (we use 24G P40), if you encounter OOM, please consider

       reduce batch-size or swith test (25,000) to train (1,998).


    e. seq_gen (compiled from seq_gen.cpp) is responsible for sampling w.r.t P, which is called within python script.




    Advanced AI Lab, Shanghai Jiao Tong University


    Fei Wang PhD@SJTU


Dataset Files

You must be an IEEE Dataport Subscriber to access these files. Login or subscribe now. Sign up to be a Beta Tester and receive a coupon code for a free subscription to IEEE DataPort!

Embed this dataset on another website

Copy and paste the HTML code below to embed your dataset:

Share via email or social media

Click the buttons below:

[1] Fei Wang, "IS-RNN", IEEE Dataport, 2018. [Online]. Available: Accessed: Dec. 09, 2019.
doi = {10.21227/H2V953},
url = {},
author = {Fei Wang },
publisher = {IEEE Dataport},
title = {IS-RNN},
year = {2018} }
AU - Fei Wang
PY - 2018
PB - IEEE Dataport
UR - 10.21227/H2V953
ER -
Fei Wang. (2018). IS-RNN. IEEE Dataport.
Fei Wang, 2018. IS-RNN. Available at:
Fei Wang. (2018). "IS-RNN." Web.
1. Fei Wang. IS-RNN [Internet]. IEEE Dataport; 2018. Available from :
Fei Wang. "IS-RNN." doi: 10.21227/H2V953