6 ms·

“AI infused at every layer“ Okay… What does that even mean? And why does the company think we want that? I love me some AI, but right now it’s getting put on

“AI infused at every layer“ Okay… What does that even mean? And why does the company think we want that? I love me some AI, but right now it’s getting put on everything like Nutella at a hipster eatery.

icepat 2y ago
Most likely more effective data harvesting. Individual user level AI isn't the end goal I don't think. It's just a nice byproduct. This just means they can embed more data collection, and use your device to crunch the numbers for them.
- kebokyo 2y ago
  This is the big thing I’m worried about with the AI revolution. If AI is being baked into your OS, where does usable training data end and your sensitive data start? Recall is pinkie promising that it won’t record sensitive data but at the same time it’s also saying “if your application doesn’t protect it well we mayyy see it and record it, whoops”. I can bet you 120% Recall is just a way for Microsoft to collect training data on millions of users every day. It is a privacy nightmare. But I don’t think the average consumer will care. Privacy died a long time ago.
lolinder 2y ago
The problem is that "AI" isn't a feature. It's not a product. It's a pretty broad category of technologies that can be used to build features and products. "AI infused at every layer" is like saying "we build our software with the best Agile(tm) practices". The customer doesn't care how you built the software, the customer cares what your software will do for them, and all "AI" means is "we probably have a deep learning inference engine somewhere under the hood, but no promises".
- lIIllIIllIIllII 2y ago
  That's like saying audio playback isn't a feature or a product.
  - lolinder 2y ago
    No, it's more like saying "media" isn't a feature or a product. It's not. It's a very vague word that could mean any number of loosely related things that do form part of many different features and products.
    - foobar1962 2y ago
      or, like saying "media ready"
    - deanCommie 2y ago
      And that was totally a thing in the early 2000's Companies were selling "Media PC's", and Windows ME stood for Millennium Edition or Media Edition depending on who you asked. And people think the word "content" is overused now...
nikanj 2y ago
We don’t want that. Investors want that.
grrowl 2y ago
it's a "fundamental transformation of the OS itself", you see
- CoastalCoder 2y ago
  Probably true. The new win64 API will have just one entry point: LPSTR do_thing(LPSTR prompt); E.g.: if (do_thing("get a mutex or whatever it's called") == do_thing("What's the happy return code for getting a mutex?") { ... }
  - emestifs 2y ago
    Someone someday may actually ship code like this into production. Horrifying to think about. For some reason this reminds me of trying to grow plants with Brawndo https://www.youtube.com/watch?v=kAqIJZeeXEc https://www.youtube.com/watch?v=kAqIJZeeXEc
  - deleted 2y ago
    [deleted]
Quiark 2y ago
I think it means many of us will not use Windows ever again.
- rstat1 2y ago
  I was already there when Windows 11 came along and blocked my perfectly functional 4 or 5 yr old HW from running it (even though the pre-releases ran just fine). This "AI" bullshit is just another nail in the already nailed shut coffin.
binkHN 2y ago
[flagged]
jiggawatts 2y ago
In practice it just means that the processor includes a matrix multiply unit somewhere. For example, some of the latest Intel Xeon CPUs have 8x 1KB registers that can perform only one operation: matrix multiply (and accumulate results). See: https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions Generally this is very similar to the Vista-era Microsoft requirements for PC manufacturers, where it was a logo requirement to include a GPU that could be used to accelerate composition in the desktop window manager, such as transparency effects and blur. (Prior to that, low-end PCs had "graphics cards" that were fixed pipeline and not programmable /general purpose.) Now Microsoft is forcing PC manufacturers to do the same kind of thing, but instead of a GPU it's now an "NPU" that they have to include. This can be a CPU instruction set, a co-processor, or a GPU baseline capability. The requirement is 40 teraoperations per second. IMHO, 40 TOPS is way too low, and doesn't focus enough on memory bandwidth. Also, that 40 is total across CPU+GPU+NPU, which means in practice it'll require fiddly optimisation to get anywhere near that level of performance. Windows Vista had the same issue, where many low-end laptops especially would struggle and spin up their GPU fans just from desktop workloads, let alone gaming...
- samspenc 2y ago
  "40 TOPS is way too low", I'm curious about this, from what I've read, Apple's M3 maxes out at 18 TOPS, and the newer M4 at 38 TOPS, so to me it sounds like even the entry-level Copilot+ PC is going to beat Apple's M3/M4 family. Am I misunderstanding.
  - wmf 2y ago
    It's possible that Microsoft's target is too low and Apple's performance is also too low. However when you find yourself saying that the entire industry is wrong you might want to stop and think.
    - jiggawatts 2y ago
      An NVIDIA 4070 puts out 836 "AI TOPS" using 200 watts of power. So the 40 TOPS target is about 10 watts of power draw equivalent, assuming it uses a similar silicon logic tech to what NVIDIA used. With a more modern process, this is about 5 watts. For an ultra-mobile tablet or laptop, this is... reasonable, I suppose. For a desktop, it's quite a bit behind the current-gen tech, let alone "the future". Like I was saying, this is aiming for where the puck was, not where the puck will be. It's the same thing as Vista GPU requirements. Vendors will do the bare minimum that ticks the checkbox, but in practice it'll be useless garbage. Modern AIs require > 1 TB/s memory bandwidth, > 128 GB memory capacity, and > 1,000 TOPS of compute to be really usable locally, not just technically capable of "running".
deleted 2y ago
[deleted]
numpad0 2y ago
Realistically, this is an announcement that Windows 12 will have a double click gguf runner and requisite backend as a feature.
- abcdefg12 2y ago
  Onnx, which seems to be awq

icepat 2y ago

Most likely more effective data harvesting. Individual user level AI isn't the end goal I don't think. It's just a nice byproduct. This just means they can embed more data collection, and use your device to crunch the numbers for them.

kebokyo 2y ago
This is the big thing I’m worried about with the AI revolution. If AI is being baked into your OS, where does usable training data end and your sensitive data start? Recall is pinkie promising that it won’t record sensitive data but at the same time it’s also saying “if your application doesn’t protect it well we mayyy see it and record it, whoops”. I can bet you 120% Recall is just a way for Microsoft to collect training data on millions of users every day. It is a privacy nightmare. But I don’t think the average consumer will care. Privacy died a long time ago.

lolinder 2y ago

The problem is that "AI" isn't a feature. It's not a product. It's a pretty broad category of technologies that can be used to build features and products. "AI infused at every layer" is like saying "we build our software with the best Agile(tm) practices". The customer doesn't care how you built the software, the customer cares what your software will do for them, and all "AI" means is "we probably have a deep learning inference engine somewhere under the hood, but no promises".

lIIllIIllIIllII 2y ago
That's like saying audio playback isn't a feature or a product.
- lolinder 2y ago
  No, it's more like saying "media" isn't a feature or a product. It's not. It's a very vague word that could mean any number of loosely related things that do form part of many different features and products.
  - foobar1962 2y ago
    or, like saying "media ready"
  - deanCommie 2y ago
    And that was totally a thing in the early 2000's Companies were selling "Media PC's", and Windows ME stood for Millennium Edition or Media Edition depending on who you asked. And people think the word "content" is overused now...

nikanj 2y ago

We don’t want that. Investors want that.

grrowl 2y ago

it's a "fundamental transformation of the OS itself", you see

CoastalCoder 2y ago
Probably true. The new win64 API will have just one entry point: LPSTR do_thing(LPSTR prompt); E.g.: if (do_thing("get a mutex or whatever it's called") == do_thing("What's the happy return code for getting a mutex?") { ... }
- emestifs 2y ago
  Someone someday may actually ship code like this into production. Horrifying to think about. For some reason this reminds me of trying to grow plants with Brawndo https://www.youtube.com/watch?v=kAqIJZeeXEc https://www.youtube.com/watch?v=kAqIJZeeXEc
- deleted 2y ago
  [deleted]

Quiark 2y ago

I think it means many of us will not use Windows ever again.

rstat1 2y ago
I was already there when Windows 11 came along and blocked my perfectly functional 4 or 5 yr old HW from running it (even though the pre-releases ran just fine). This "AI" bullshit is just another nail in the already nailed shut coffin.

binkHN 2y ago

[flagged]

jiggawatts 2y ago

In practice it just means that the processor includes a matrix multiply unit somewhere. For example, some of the latest Intel Xeon CPUs have 8x 1KB registers that can perform only one operation: matrix multiply (and accumulate results). See: https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions Generally this is very similar to the Vista-era Microsoft requirements for PC manufacturers, where it was a logo requirement to include a GPU that could be used to accelerate composition in the desktop window manager, such as transparency effects and blur. (Prior to that, low-end PCs had "graphics cards" that were fixed pipeline and not programmable /general purpose.) Now Microsoft is forcing PC manufacturers to do the same kind of thing, but instead of a GPU it's now an "NPU" that they have to include. This can be a CPU instruction set, a co-processor, or a GPU baseline capability. The requirement is 40 teraoperations per second. IMHO, 40 TOPS is way too low, and doesn't focus enough on memory bandwidth. Also, that 40 is total across CPU+GPU+NPU, which means in practice it'll require fiddly optimisation to get anywhere near that level of performance. Windows Vista had the same issue, where many low-end laptops especially would struggle and spin up their GPU fans just from desktop workloads, let alone gaming...

samspenc 2y ago
"40 TOPS is way too low", I'm curious about this, from what I've read, Apple's M3 maxes out at 18 TOPS, and the newer M4 at 38 TOPS, so to me it sounds like even the entry-level Copilot+ PC is going to beat Apple's M3/M4 family. Am I misunderstanding.
- wmf 2y ago
  It's possible that Microsoft's target is too low and Apple's performance is also too low. However when you find yourself saying that the entire industry is wrong you might want to stop and think.
  - jiggawatts 2y ago
    An NVIDIA 4070 puts out 836 "AI TOPS" using 200 watts of power. So the 40 TOPS target is about 10 watts of power draw equivalent, assuming it uses a similar silicon logic tech to what NVIDIA used. With a more modern process, this is about 5 watts. For an ultra-mobile tablet or laptop, this is... reasonable, I suppose. For a desktop, it's quite a bit behind the current-gen tech, let alone "the future". Like I was saying, this is aiming for where the puck was, not where the puck will be. It's the same thing as Vista GPU requirements. Vendors will do the bare minimum that ticks the checkbox, but in practice it'll be useless garbage. Modern AIs require > 1 TB/s memory bandwidth, > 128 GB memory capacity, and > 1,000 TOPS of compute to be really usable locally, not just technically capable of "running".

deleted 2y ago

[deleted]

numpad0 2y ago

Realistically, this is an announcement that Windows 12 will have a double click gguf runner and requisite backend as a feature.

abcdefg12 2y ago
Onnx, which seems to be awq