During decoding the processing time increases linearly with the length of the audio input segment.
the largest time contribution is due to the computations inside the network (He and Sun, 2015), which with a single convolution layer can be performed in negligible time for single utterances.