Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)
anemll
30 points
12 comments
October 17, 2026
Related Discussions
Found 5 related stories in 98.9ms across 9,789 title embeddings via pgvector HNSW
- Qwen 3.8 27B available on Cerebras at 1500 tokens/s altertable · 509 pts · September 03, 2026 · 68% similar
- Qwen 3.8 Max linzhangrun · 26 pts · July 19, 2026 · 66% similar
- Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU pich · 37 pts · August 17, 2026 · 65% similar
- I made my iPhone a second GPU for my MacBook-Qwen 3.8 27B prefills 29–44% faster thunderbong · 24 pts · October 03, 2026 · 65% similar
- Running Qwen3.8 at 255 tok/s on a single 5090 Ir0n · 12 pts · October 13, 2026 · 64% similar
Discussion Highlights (4 comments)
spacecoder2
Anyone know how hot an iPhone gets running the Neural Engine flat out for 10 minutes?
analog_daddy
Hi. Could anyone share any recommendations (hopefully with actual links) for App Store apps to try running such local models? I’m not really interested in side-loading since it currently seems (to me) more of an hassle than it’s worth. App Store search is broken in a sense that there might be so many apps claiming to be local but once you read reviews, it’s just another scam where it’s it’s usually sending your query to remote server.
Retr0id
OP, asking for upvotes is against the HN guidelines: https://x.com/anemll/status/2111531629529456771
createful
Not Qwen, Bonsai. Although it is based on Qwen, and achieves similar scores, it is not the "full" Qwen. Last I remember, when trying to run a model above 1-2 GB on iOS, it would refuse to do so, at least from Locally AI. iOS only allows apps to use about 1-2 GB of RAM before being killed. Though using their Neural APIs might bypass this.