This is likely because it is easier to learn the parameters of the image prediction model that has fewer parameters (8.192 million for VGG-19 vs. 4.096 million for Inception-V3 and ResNet-50).
it is not clear why there is such a pronounced difference between the Inception-V3 and ResNet-50 models 4 .