A highway connection is a gating mechanism that combines the current and previous layer outputs, which can prevent the problem of vanishing/exploding gradients (Srivastava et al., 2015).
in networks with highway connections, we replace Eq.