Gadgets · by Rachel Tan · 2026-10-10
Apple Silicon unified memory is quietly the best deal for running local LLMs right now
Not an Apple fanboy generally but the unified memory architecture on M-series chips makes running 30-70B parameter models locally way more accessible than assembling a multi-GPU rig, since you're not bottlenecked by VRAM the same way. Running quantized models through Ollama/LM Studio on a maxed out Mac Studio feels like the sweet spot for privacy-conscious local workflows. Tradeoff is obviously raw throughput vs a real GPU cluster. Anyone doing serious local inference work want to share their setup and real tokens/sec numbers?
0 sparks · 8 comments
Comments
Camila Rocha · 0 likes
M3 Ultra Mac Studio here running 70b quantized models, getting around 8-10 tok/s which is usable but not fast, its more about privacy for me than speed
Rapid Orbit · 0 likes
unified memory is great until you actually need to train/fine-tune something, then the lack of real tensor core throughput hurts a lot more
Ananya Rao · 0 likes
yeah inference vs training are totally different workloads here, apple silicon is an inference device not a training rig
Lucas Silva · 0 likes
went the other way, built a used 3090 rig for way less money with way more raw speed, just dont get the unified memory flexibility for huge models
Alma Nilsson · 0 likes
the power efficiency difference is honestly underrated too, my mac studio sips power compared to what my old gpu rig used to pull
Noor Janssens · 0 likes
ollama + a mac mini has been my 'good enough' setup for small models, dont need the studio unless your going for the bigger weights
Mina Chowdhury · 0 likes
quantization quality loss is real though, a 70b model at q4 isnt the same as the full precision version even if the specs look fine on paper
Haruto Sato · 0 likes
this is basically why local ai hobbyists have been buying up used mac studios, resale prices on the 192gb models went up not down lol