Is the new Apple M5 actually worth it if all you care about is local LLM inference?

Lucía García · 0 likes

wait for the ultra, every single time the base/pro/max chips get announced people rush to buy and then the ultra studio drops 4 months later and makes it look silly

Hamza Malik · 0 likes

memory bandwidth is genuinely the bottleneck for token generation speed on these chips, and yes the improvement is real, i've run quantized 70b models and its noticeably snappier than my M3

Cozy Noodle · 0 likes

do you have actual tok/s numbers? curious how it compares to a used 4090 setup for the same model size

Camila Rocha · 0 likes

for pure inference a used 3090/4090 rig is still way more cost effective per token, apple silicon wins on power draw and silence, not raw price performance

Paper Atlas · 0 likes

the unified memory capacity is the real unlock though, running models that simply wont fit on consumer nvidia VRAM without serious offloading tricks

Ananya Rao · 0 likes

i went the mac studio route specifically because i wanted a quiet machine in my living room, the performance per watt argument genuinely matters for some setups

Ngozi Bello · 0 likes

honestly the software ecosystem (MLX etc) maturing has done more for apple silicon LLM performance over the last year then the hardware jumps themselves

Nour Ali · 0 likes

if you're doing any actual fine-tuning not just inference, nvidia's CUDA ecosystem still has way better tooling support, worth factoring in