The best performing results were achieved with two hidden layers (400 and 500 nodes respectively), tanh for activation function, and learning rate of 0.001 in gradient decent with early stopping.
the networks could not provide superior results than the SVM regressors.