QuoteAnd if the M6 is not powerful enough or you need more than 32 GB RAM, Apple also offers the Mac mini with the M5 Pro SoC.
This is good I guess, but the not-improvement from previous M5 to more than 32 GB RAM/unified memory is a disappointment, because if one wants to run SOTA model huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/blob/main/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf[1] -- and this model would run fast enough on this mini-PC, if it only had 8 GB more RAM/unified memory --, this is how many context tokens one can fit (for this MTP variant, context will be less, because the MTP head also consumes memory):
Quote from: reddit.com/r/LocalLLaMA/comments/1sq94qx/is_anyone_getting_real_coding_work_done_with.. I've come to the conclusion that (1) 32768 is the biggest context I can get away with in an adequately smart model, and (2) it just ain't enough.
Calculating available context: 16,384 [KV cache context tokens per GB]*(32 [GB total memory] - 6 to 8 [GB for the OS] - [quant filesize])[2]:
For huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF -> UD-Q4_K_XL.gguf:
~34,400 context tokens = 16,384*(32-7-22.9) (confirms the quote)
Hopefully the M7 will come with more than up to 32 GB RAM, 40 GB would be a good start, if APPLE considers 48 GB to eat too much into M Pro.
[1] artificialanalysis.ai/?models=qwen3-8-27b%2Cqwen3-8-27b-medium%2Cqwen3-8-27b-low%2Cqwen3-8-27b-non-reasoning%2Cqwen3-6-35b-a3b%2Cqwen3-6-35b-a3b-non-reasoning&intelligence=artificial-analysis-intelligence-index
[2] reddit.com/r/Qwen_AI/comments/1vo8pjz/qwen3827b_kv_cache_works_out_to_64_kibtoken_so/