Problems with inference
#38
by
Kirill200223
- opened
I'm trying to load the model in the simplest way through torch, but I'm always stuck on
Loading weights: 778it [04:05, 3.17it/s, Materializing param=lm_head.weight]
the graphics cards are loaded, but the process hangs endlessly in this place. Moreover, the same code works without errors in 1 out of every 10 cases. Setup: 4 A100 80GB
I've tried running it on different servers in different environments, and the problem is the same everywhere. It's like accelerate freezes in deadlock.