For the matrix sizes involved (> 500×1, 000), the time complexity of matrix product is linear in each dimension, both on CPU (Intel MKL 5 ) and GPU (CuBLAS), with a 10X speedup for GPU (Nvidia K40) compared to CPU (Intel Xeon E5-2680).
the speed trade-off applies to both CPU and GPU hardware, albeit with a different time scale.