I work as as a researcher and am kind of new to neural networks. I have an RNN (1e4 x 1e4 network) that I would like to train in either MATLAB or Julia.

One option I considered is writing my own code for Hessian-free optimization, but the implementational details are really, really hard to figure out.

I am aware there is a Theano or TF implementation of HFO but I I am primarily interested in having the code in MATLAB/Julia.

Also, are there better/alternative techniques than Hessian-free optimization for training RNN's ?