AI Lab · by Tiny Fern · 2026-10-11
Running a full local AI dev workflow on Apple Silicon with Ollama - no cloud API calls at all
Spent the weekend setting up a fully offline dev workflow: Ollama for local inference, a small quantized coding model for autocomplete, and a bigger model for architecture questions, all on an M-series Mac. Latency is obviously worse than hitting an API but the privacy win (no code leaving the machine) and zero recurring cost made it worth it for personal projects. Anyone else gone fully local, and which quantization level actually holds up for real coding work vs toy demos?
0 sparks · 9 comments
Comments
Tomás Gómez · 0 likes
Q4_K_M is the sweet spot imo, Q8 barely improves quality but eats way more ram and tokens/sec drops a lot
Aoife Murphy · 0 likes
did this for 3 months then went back to cloud apis. local models are great until you need something genuinely hard, then the gap shows
Sleepy Orbit · 0 likes
yeah local is good for 'autocomplete plus' not for architecture decisions in my experience
Grace Parker · 0 likes
the privacy angle is underrated, work at a place that literally cannot send proprietary code to any external api, local is the only option for us
Nusrat Akter · 0 likes
unified memory on apple silicon is what makes this actually usable, tried similar setup on a windows laptop with dedicated gpu and ran out of vram constantly
Tola Ibrahim · 0 likes
how are you handling tool calling locally? thats where i always hit a wall, smaller models just arent reliable at structured outputs
Tiny Noodle · 0 likes
honestly still a pain point, i constrain the grammar with a json schema and it helps but not perfect
Solar Fern · 0 likes
battery life absolutely tanks running inference locally for hours, worth mentioning if youre not plugged in
Thandi Mokoena · 0 likes
this is the setup i want to build but the config and tooling is still so fragmented compared to just hitting an api endpoint