Training Loss increases

[abhitrip]

Is the loss being output on trainLSTM_1.py , the real train loss or just the loss of some random epoch ?
Cause My training loss seems to increase after 2 epochs ...
FYI : I have used the glove vectors as the word vectors.

[avisingh599]

Does it increase consistently, or only for 1-2 epochs?

[abhitrip]

For the first 4 epochs its like :
215424/215375 [==============================] - 418s - train loss: 3.7656
215424/215375 [==============================] - 432s - train loss: 3.7264
215424/215375 [==============================] - 439s - train loss: 3.7595
215424/215375 [==============================] - 442s - train loss: 3.7989
I have run the code for 20 epochs , and it is continuously increasing

[avisingh599]

Does not seem right. Try reducing your learning rates, and see what you get. You might also be interested in trying a bunch of different learning rates in parallel, and see which one converges the fastest.

[abhitrip]

I tried reducing the learning rate and yes now the loss decreases generally.
Can you give me a fair idea of what was the loss you were getting after around 100 epochs, cause now the losses are decreasing , but not by much ...

[avisingh599]

I did this project a long time ago, and unfortunately I don't have access to most of the data associated with this project anymore (I graduated undergrad, and do not have access to my machine there anymore).

I remember getting competitive performance on the benchmark around 50-60 epochs, so you might want to try that.

[abhitrip]

Ok , thanks :) let me run a couple more epochs and I will let you know