What? Kuo never said Apple will drop unified memory. By separate CPU/GPU Kuo likely just means Apple will use chiplets. Using TSMC's SoIC-mH the 2.5D chiplets still be a single chip with unified memory

Quote from: Derek B on December 30, 2024, 21:02:48Unified memory is not about the physical location of the memory chips. And it not the same as shared memory. Shared memory is the CPU and GPU using the same memory but separately. Once part of the memory is assigned to the GPU, the CPU can't operate on that portion directly, ther still must be copying between CPU and GPU portions.

Unified memory refers to the memory management units being coherent, so that they can both use the same data without the need to copy.

That coherency between the two MMUs will remain regardless of where the memory chips end up.

Yup. Apple wasn't forced to abandon unified memory in an Ultra design both sides of the interconnect. There are a lot of interesting implementation variations when making a separate GPU die in terms of which die the IO goes onto (separate IO die, CPU die, or GPU die, or both CPU/GPU die), but no particular reason Apple will need to abandon unified memory.

Unified memory is not about the physical location of the memory chips. And it not the same as shared memory. Shared memory is the CPU and GPU using the same memory but separately. Once part of the memory is assigned to the GPU, the CPU can't operate on that portion directly, ther still must be copying between CPU and GPU portions.

Unified memory refers to the memory management units being coherent, so that they can both use the same data without the need to copy.

That coherency between the two MMUs will remain regardless of where the memory chips end up.

Advantages of unified (shared) memory are avoided VRAM size limit and faster bandwidth for interaction involving both RAM- and VRAM-like memory use. An advantage of a separate dGPU can be availability of CUDA, RT and Tensor cores. Whether AI inference profits from a particular system, memory, GPU and possibly networking structure and from the adforementioned particular types of cores depends on the software code and algorithms. Avoiding unified memory might, or might not, accelerate different softwares of AI inference. Adding (and licensing) CUDA, RT and Tensor cores would greatly accelerate quite a few softwares of AI inference - those currently profiting from Nvidia dGPUs. If Apple really wants to go AI with M5, it must offer those types of cores together with equivalents of CUDA, CuDNN and TensorRT libraries of comparable algorithmic quality because this alone is responsible for up to the speed factor 2.95. However, Apple hating license fees is unlikely to go all in at AI and rather will try to let PR replace much of then still missing actual speed. With instead reinventing AI-specific cores and libraries for them, Apple will need 20 years to catch up to Nvidia's AI capabilities and then AI softwares must be rewritten for Apple silicon.