Quote from: Mr Majestyk on February 27, 2024, 04:04:10You can also run AI software on the cpu, npu and gpu simultaneously.

If it is the same software (instance), it needs to support using multiple chips and possibly different kinds of cores. Some such softwares scale almost proportionally to numbers of (comparable) cores distributed to several chips.

Quote from: RobertJasiek on February 26, 2024, 16:45:51

Quote from: davidm on February 26, 2024, 16:03:45It's hard to understand how people would not see this as useful

For me, the question rather is: how useful? If I want AI, I use my Nvidia dGPU instead of a tiny NPU with a low TDP budget. Thereby, I get speed presumably dozens or hundreds of times faster and have more storage incl. VRAM.

One application of an NPU that does convince me in principle is fan control for the sake of lower noise at sufficient cooling. I suppose it also requires suitable drivers and firmwares.

The NPU will draw far less power as it's ASIC designed for this sole purpose. On a laptop the NPU's will get more useful as they get more powerful and offer better battery life. You can also run AI software on the cpu, npu and gpu simultaneously.

Quote from: RobertJasiek on February 26, 2024, 20:18:39NPUs are about to become a thing for modest AI use in smartphones

this will not age well too, just notifying you ahead

google gemini isn't running and wasn't trained on nvidia anything
asics in the end of the day are always faster and are more efficient that generalized hardware

Quote from: davidm on February 26, 2024, 18:34:51You're not going to take your NVidia DGPU out of the basement.

Hehe, sure, mine is in a heavy desktop. Others might have a notebook with Nvidia dGPU (or, you know, an Apple, or - less suitable for that purpose - AMD or Intel dGPU).

NPUs are about to become a thing for modest AI use in smartphones & Co. (As you know, I do not have a smartphone. Many others do, and some of them will use NPUs in them.)

QuoteVRAM is not important for everyday models when you have fast unified RAM.

We know. (It is my preference to use x64, and the AI I use prefers Nvidia.)

Quote from: RobertJasiek on February 26, 2024, 16:45:51

Quote from: davidm on February 26, 2024, 16:03:45It's hard to understand how people would not see this as useful

For me, the question rather is: how useful? If I want AI, I use my Nvidia dGPU instead of a tiny NPU with a low TDP budget. Thereby, I get speed presumably dozens or hundreds of times faster and have more storage incl. VRAM.

One application of an NPU that does convince me in principle is fan control for the sake of lower noise at sufficient cooling. I suppose it also requires suitable drivers and firmwares.

You're not going to take your NVidia DGPU out of the basement. VRAM is not important for everyday models when you have fast unified RAM.

yeah robert every single of your headcanon 'ideas' about ai never aged well and you keep mindlessly double-downing.

Quote from: davidm on February 26, 2024, 16:03:45It's hard to understand how people would not see this as useful

For me, the question rather is: how useful? If I want AI, I use my Nvidia dGPU instead of a tiny NPU with a low TDP budget. Thereby, I get speed presumably dozens or hundreds of times faster and have more storage incl. VRAM.

One application of an NPU that does convince me in principle is fan control for the sake of lower noise at sufficient cooling. I suppose it also requires suitable drivers and firmwares.

It's hard to understand how people would not see this as useful, and just wait until local databases (RAG) becomes a thing. Yet people, especially in the x86 world where they're entrenched in their idea anything non-classic-socketed-RAM-x86 is a personal attack, are pushing back as if their 10% generational improvements are all that counts. I guess that's what happens when aging gamers are your main audience.