Ollama Now Runs Faster on Macs Thanks to Apple's MLX Framework

This is going to be some serious cash flow incoming for Apple in this year.

I think this could be a major business for Apple - it’s way cheaper for a small business to buy a powerful Mac and run qwen 3.5 than pay for an enterprise license for a frontier model - and you don’t need to worry about privacy issues.

On device is definitely gonna be the future.

I can’t help but wonder if Apple looked ahead and foresaw this when developing the M series, or if they’ve lucked into it.

This is going to be some serious cash flow incoming for Apple in this year.

M-Series chips at work😎

Just tested the new Ollama MLX runner in CLI via terminal (0.19.0 preview) on my Mac mini M2 Pro 32GB, running qwen3.5:35b-a3b-coding-nvfp4.

it works, and it's noticeably faster than standard non-MLX models! Standard Qwen3.5 35b is not usable on my Mac, this one yes! It’s incredible!

What works well:
- The model loads and runs without issues on 32GB
- with /set nothink it’s blazing fast
- Token generation speed is much higher than equivalent non-MLX models
- RAM pressure stays in the green/yellow zone during normal use

Limitations:
- 32GB is the hard ceiling — the model itself takes ~20GB, leaving ~12GB for KV cache. Short sessions are fine; long context pushes into swap
- Can't comfortably use it as a backend for agentic frameworks (like OpenClaw) where the context grows large — hits the 32GB wall quickly

Bottom line: great for interactive chat sessions via ollama run, but this way it’s fun just for a session, not useful. For production agentic use or long context, you'd want 48GB+. Looking forward to seeing this on an M5 Max.

As someone who downloads and experiments with everything possible…

There is a lot of delusion in this thread. Local language models below 100 billion parameters are quite useless. Even 100 billion parameters is considered the weak side. Fun to play with for a while but boredom and frustration sets in quickly.

So what happens is they want the next model…and then the next one…and then the next one…falsely believing their 16GB or 32GB machine will one day have the holy grail of small and powerful local language model.

But it doesn’t happen. The models keep growing and aside from being memory hungry the most important thing that makes them useable is memory bandwidth.

The top 5 language models in the world are all over a trillion parameters and what makes them useful and responsive is that they respond quickly and have GPU with over a terabyte of bandwidth.