
LSTM: How to Train Neural Networks to Write Like Lovecraft - strikingloo
http://www.datastuff.tech/machine-learning/lstm-how-to-train-neural-networks-to-write-like-lovecraft/
======
strikingloo
Hey guys, I'm the writer. As you can see from the post, I'm still very much
learning.

What I want the most from this site is for more experienced people to help me
out with some of my questions.

Here they come:

\- Can you use Batch Normalization (the one from tf.keras) on an LSTM layer?
Or will it break the model?

\- How do you deal with extremely infrequent words if you do a word-based LSTM
(with a one-hot encode of each word in the corpus?)? Do you remove them?
Replace them? Cluster them?

\- Do you think there's any other architecture that would've had better
results -while still not taking too long to train-?

~~~
srean
> Do you think there's any other architecture that would've had better results
> -while still not taking too long to train-?

Yes! A 5th or 6th order word level Markov chain with smoothing.

~~~
strikingloo
Thanks! I hadn't thought of doing Markov Chains since I was focusing on
RNN/LSTMs, but I think I'll try that for the next article, see how that goes.

