AI Lab · by Tiago Costa · 2026-10-12
DeepSeek and Qwen are closing the gap faster than I expected on coding benchmarks
Ran a local Qwen model through our internal eval suite (private, not leaked into training data) and it handled multi-step refactors nearly as well as last year's frontier closed models. The open-weight ecosystem is moving stupid fast. If you're running local models on Apple Silicon or with Ollama, what's your current daily driver for actual dev work vs just tinkering?
0 sparks · 9 comments
Comments
Camila Rocha · 0 likes
qwen2.5-coder on my m3 max is genuinely good enough for 80% of my day to day now, only reach for cloud models on genuinely hard problems
Kabir Khan · 0 likes
ollama + qwen coder 32b quantized runs surprisingly smooth on 64gb unified memory, dont even need the full precision weights for most tasks
Irene López · 0 likes
whats ur tok/s on that setup? im getting like 18 on my m2 ultra which feels slow
Luciana Vargas · 0 likes
deepseek v3 architecture choices (MoE routing specifically) are genuinely clever, more labs should be publishing papers like this instead of just model cards
Sleepy Sparrow · 0 likes
local models are great until you need long context, thats still where cloud wins by a mile for me
Urban Otter · 0 likes
privacy is the actual reason i run local, not performance. cant send client code to any api no matter how good
Aria Thompson · 0 likes
same, legal wont even let us consider cloud inference for half our codebase
Linh Tran · 0 likes
the pace of open weight releases this year has been insane, feels like every month theres a new 'best open model' announcement
Noah Miller · 0 likes
anyone benchmarked these against real prod bugs instead of leetcode style problems? thats where i still see the biggest gap