To control for this confound we also evaluated a version of our model that enforces block-sparsity throughout, but retains the use of componentwise addition at the inputs.
its performance is ultimately not quite as good as using a character LSTM.