In machine learning, a recurrent neural network (RNN or LSTM) is a class of neural networks that have successfully been applied to Natural Language Processing. In this tutorial, I will explain how to build an RNN model with LSTM or GRU cell to predict the prices of the New York Stock Exchange.
The implementation of the network has been made using TensorFlow High-level API like Dataset API to feed data into model and Estimators API to train and predict model. I will describe the following steps: dataset creation, RNN training and prediction.
Download Data
The dataset can be downloaded from Kaggle. Data found on Kaggle is a collection of CSV files. You don’t have to do any preprocessing. You can directly load the data into a Pandas DataFrame.Stock prices come in several different flavours. They are, Open, Close, High and Low.
import numpy as np import pandas as pd import sklearn import sklearn.preprocessing import tensorflow as tf from matplotlib import pyplot as plt tf.logging.set_verbosity(tf.logging.INFO) tf.logging.log(tf.logging.INFO, "TensorFlow version " + tf.__version__) DATA_PATH = 'data/nyse/prices-split-adjusted.csv' df = pd.read_csv(DATA_PATH, index_col=0) SYMBOL_NAME = 'GOOG' df_stock = df[df.symbol == SYMBOL_NAME].copy() df_stock.drop(['symbol'], 1, inplace=True) df_stock.drop(['volume'], 1, inplace=True)
Data scaling
Most common activation functions of the network’s neurons such as tanh or sigmoid are defined on the [-1, 1] or [0, 1] interval respectively. Scaling can be easily accomplished in Python using sklearn’s MinMaxScaler.
def scale_data(df):
min_max_scaler = sklearn.preprocessing.MinMaxScaler()
df['open'] = min_max_scaler.fit_transform(df.open.values.reshape(-1, 1))
df['high'] = min_max_scaler.fit_transform(df.high.values.reshape(-1, 1))
df['low'] = min_max_scaler.fit_transform(df.low.values.reshape(-1, 1))
df['close'] = min_max_scaler.fit_transform(df.close.values.reshape(-1, 1))
return df
df_stock_norm = df_stock.copy()
df = scale_data(df_stock_norm)
Target Data
We will give it a sequence of stock prices and ask it to predict the next day price using GRU cells. We put our sequence of stock prices on the inputs. It will produce some kind of number on the output. To teach it we force a sequence on the outputs which is the same sequence shifted by one number. This is what we will be teaching.
SEQLEN = 20 df_input = df df_target = df.shift(-1) df_input = df_input[:-2].values df_target = df_target[:-2].values X = np.reshape(df_input, (-1, SEQLEN, 4)) Y = np.reshape(df_target, (-1, SEQLEN, 4))
Split Data
The dataset was split into training and test data. The training data contained 80% of the total dataset. The data was not shuffled but sequentially sliced.
train_split = 0.8 num_data = X.shape[0] num_train = int(train_split * num_data) x_train = X[0:num_train] y_train = Y[0:num_train] y_test = Y[num_train:] x_test = X[num_train:]
Dataset Input Function
In this tutorial, we will use Dataset API to input data in model and Estimator API to train model. You need to provide an input_fn to read your data. You provide a function that returns inputs and labels. The inputs are a dictionary of all your inputs and the labels is a tensor.
def train_input():
dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
dataset = dataset.repeat()
dataset = dataset.shuffle(SHUFFLE_SIZE)
dataset = dataset.batch(BATCHSIZE)
samples, labels = dataset.make_one_shot_iterator().get_next()
return samples, labels
def test_input():
dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test))
dataset = dataset.repeat(1)
dataset = dataset.batch(BATCHSIZE)
samples, labels = dataset.make_one_shot_iterator().get_next()
return samples, labels
Create LSTM Model
There is a full API for working with RNN in TensorFlow. It starts with a GRU cell. This defines all the weights and biases. The only parameter you need to specify is this internal size of the vectors in this cell.
RNN_CELLSIZE = 80
N_LAYERS = 2
DROPOUT_PKEEP = 0.7
def model_rnn_fn(features, labels, mode):
batchsize = tf.shape(features)[0]
seqlen = tf.shape(features)[1]
cells = [tf.nn.rnn_cell.GRUCell(RNN_CELLSIZE) for _ in range(N_LAYERS)]
cells[:-1] = [tf.nn.rnn_cell.DropoutWrapper(cell, output_keep_prob=DROPOUT_PKEEP) for cell in cells[:-1]]
If you want to deep neural network. Let’s stack it, there is a function for that it’s called MultiRNNCell you pass it a cell and how many times you want this cell and it creates a new cell which is a stacked cell.
cell = tf.nn.rnn_cell.MultiRNNCell(cells, state_is_tuple=False)
In TensorFlow you can unroll cell using the dynamic RNN function you give it a stacked cell that you just produced. How many times do you unroll? That will depend on the shape of X. X here my input is a sequence of the number. If I feed it with sequences of 16 numbers my network will be unrolled 16 times. what I get as an output are results? It’s all the ages on the bottom.
Yn, H = tf.nn.dynamic_rnn(cell, features, dtype=tf.float64)
Yn = tf.reshape(Yn, [batchsize * seqlen, RNN_CELLSIZE])
Yr = tf.layers.dense(Yn, 4) # Yr l[BATCHSIZE*SEQLEN, 1]
Yr = tf.reshape(Yr, [batchsize, seqlen, 4]) # Yr [BATCHSIZE, SEQLEN, 1]
Yout = Yr[:, -1, :] # Last output Yout [BATCHSIZE, 1]
loss = train_op = None
if mode != tf.estimator.ModeKeys.PREDICT:
loss = tf.losses.mean_squared_error(Yr, labels) # la bels[BATCHSIZE, SEQLEN, 1]
lr = 0.001
optimizer = tf.train.AdamOptimizer(learning_rate=lr)
train_op = tf.contrib.training.create_train_op(loss, optimizer)
return tf.estimator.EstimatorSpec(
mode=mode,
predictions={"Yout": Yout},
loss=loss,
train_op=train_op
)
you need to apply an activation layer on those to obtain my result sequence and I get also the last state H which is the state that will need to pass back in the beginning as I continue my training.
Create the Estimator
The estimator is a TensorFlow class for performing high-level model training, evaluation, and inference for our model. The model_fn argument specifies the model function to use for training, evaluation, and prediction pass it the model_rnn_fn. The model_dir argument specifies the directory where model data and checkpoints will be saved.
training_config = tf.estimator.RunConfig(model_dir="./output") estimator = tf.estimator.Estimator(model_fn=model_rnn_fn, config=training_config)
Train RNN Model
Now we’re ready to train our model, which we can do by calling train() on estimator.
estimator.train(input_fn=train_input, steps=10000)
Predictions from Trained Model
We now have a trained model. We can now use the trained model to predict time series data based on some unlabeled data. As with training, we make predictions using a single function call.
results = estimator.predict(test_input)
Yout_ = [result["Yout"] for result in results]
predict = np.array(Yout_)
actual = y_test[:, -1]
colors = plt.rcParams['axes.prop_cycle'].by_key()['color']
plt.plot(actual[:, 0], label="Actual Values", color='red')
plt.plot(predict[:, 0], label="Predicted Values", color='green', )
plt.title('stock')
plt.xlabel('time [days]')
plt.ylabel('normalized price')
plt.legend(loc='best')
plt.show()
