Kim2091/AnimeSharp
Image-to-Image
•
Updated
•
18
Gary Mitchell
echos-keeper
AI & ML interests
None yet
Recent Activity
liked
a model
3 days ago
Kim2091/AnimeSharp
liked
a model
6 days ago
FoodDesert/Boring_Embeddings
liked
a model
10 days ago
unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF
Organizations
None yet
liked
a
model
3 days ago
liked
a
model
6 days ago
liked
a
model
10 days ago
liked
a
dataset
14 days ago
upvoted
a
collection
17 days ago
liked
a
dataset
17 days ago
liked
a
model
17 days ago
liked
a
model
18 days ago
liked
a
model
20 days ago
New activity in
TeichAI/Qwen3-30B-A3B-Thinking-2507-Claude-4.5-Sonnet-High-Reasoning-Distill
21 days ago
fixed base model name
1
#1 opened 21 days ago
by
echos-keeper
New activity in
TeichAI/Qwen3-30B-A3B-Thinking-2507-Claude-4.5-Sonnet-High-Reasoning-Distill-GGUF
21 days ago
fixed base model name
#1 opened 21 days ago
by
echos-keeper
liked
a
model
21 days ago
updated
a
model
22 days ago
published
a
model
22 days ago
upvoted
a
collection
23 days ago
updated
a
model
24 days ago
published
a
model
24 days ago
liked
a
model
29 days ago
reacted to
mitkox's
post with 👀
29 days ago
Post
4128
I just threw Qwen3-0.6B in BF16 into an on device AI drag race on AMD Strix Halo with vLLM:
564 tokens/sec on short 100-token sprints
96 tokens/sec on 8K-token marathons
TL;DR You don't just run AI on AMD. You negotiate with it.
The hardware absolutely delivers. Spoiler alert; there is exactly ONE configuration where vLLM + ROCm + Triton + PyTorch + Drivers + Ubuntu Kernel to work at the same time. Finding it required the patience of a saint
Consumer AMD for AI inference is the ultimate "budget warrior" play, insane performance-per-euro, but you need hardcore technical skills that would make a senior sysadmin nod in quiet respect.
564 tokens/sec on short 100-token sprints
96 tokens/sec on 8K-token marathons
TL;DR You don't just run AI on AMD. You negotiate with it.
The hardware absolutely delivers. Spoiler alert; there is exactly ONE configuration where vLLM + ROCm + Triton + PyTorch + Drivers + Ubuntu Kernel to work at the same time. Finding it required the patience of a saint
Consumer AMD for AI inference is the ultimate "budget warrior" play, insane performance-per-euro, but you need hardcore technical skills that would make a senior sysadmin nod in quiet respect.
- 1 reply
reacted to
mike-ravkine's
post with 🔥
29 days ago
Post
2767
There is no anxiety quite like powering up 2KW of basement compute after rewiring it all. Small bit of trouble with the horizontal 3090 because I misread my motherboard manual, but otherwise so far so good.. Next we see if I've built up enough cooling to hit my target TDP on those 3-slot nvlinked cards especially. The 4-slot bridges are much easier to work with but their prices went bananas and I couldn't acquire a second, so gotta get a little creative with intakes.
- 5 replies