Networks with ReLU have a lower run time and tend to show better convergence (Talathi and Vartak 2015).
reLU has the disadvantage of dying cells (the dying reLU problem), but this can be overcome by using a variant called Leaky reLU.