10 ms·

An appeal to Apple from Anukari

405 points by humbledrone 1y ago

ArthurStacks 1y ago
[flagged]
- mac9 1y ago
  Some of them probably do... This is still a funny comment though
humbledrone 1y ago
Some folks may have seen my Show HN post for Anukari here: https://news.ycombinator.com/item?id=43873074 https://news.ycombinator.com/item?id=43873074 In that thread, the topic of macOS performance came up there. Basically Anukari works great for most people on Apple silicon, including base-model M1 hardware. I've done all my testing on a base M1 and it works wonderfully. The hardware is incredible. But to make it work, I had to implement an unholy abomination of a workaround to get macOS to increase the GPU clock rate for the audio processing to be fast enough. The normal heuristics that macOS uses for the GPU performance state don't understand the weird Anukari workload. Anyway, I finally had time to write down the full situation, in terrible detail, so that I could ask for help getting in touch with the right person at Apple, probably someone who works on the Metal API. Help! :)
- aplummer 1y ago
  Have you filed a feedback? Seems like the right next step.
  - viraptor 1y ago
    Feedback is as effective as creating a change.org petition to some politician to stop doing crimes please. You'll be lucky to get an acknowledgement that something's a real issue after months.
  - bayindirh 1y ago
    The post opens with the following TL;DR:, snipped for brevity: > It would be great if someone can connect me with the right person inside Apple, or direct them to my feedback request FB17475838 as well as this devlog entry.
    - sgerenser 1y ago
      Feedbacks often go into a black hole unless either: 1. A bunch of people file effectively the same bug report (unlikely here) 2. An individual Apple employee champions the issue internally 3. Someone makes a fuss on Twitter/X and it starts to go viral Sounds like the OP is trying to get #2 to happen, which is probably his best bet.
      - badc0ffee 1y ago
        Another trick is to schedule some Apple engineer time during WWDC, and plead your case.
        musicale 1y ago
        I was going to recommend this. They may have some suggestions of how to improve things with existing metal as well.
- TheAceOfHearts 1y ago
  I missed the Show HN, but the first thing that came to mind after seeing it was that this looks like it would lend itself well to making some very creative ASMR soundscapes with immersive multidimensional audio. I selfishly hope you or one of your users will make a demo. Congrats on the project and I hope you receive help on your Apple issues.
- bambax 1y ago
  > This is going to be a VERY LONG HIGHLY TECHNICAL post, so either buckle your seatbelt or leave while you still can. Well, I read it all and found it not too long, extremely clear and well-written, and informative! Congrats on the writing. I've never owned a Mac and my pc is old and without a serious GPU, so it's unlikely that I'll get to use Anukari soon, but I regret it very much, as it looks sooo incredibly cool. Hope this gets resolved fast!
- Dlemo 1y ago
  [flagged]
  - mensetmanusman 1y ago
    It’s technical to over half of programmers who don’t need to know these types of details about hw/sw interactions.
    - Dlemo 1y ago
      It's about 'very technical'. If you can explain the problem in one basic sentence it's not very
- my123 1y ago
  Did you try this entitlement? https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.sustained-execution https://developer.apple.com/documentation/bundleresources/en... wonder if com.apple.developer.sustained-execution also goes the other way around...
  - humbledrone 1y ago
    Thanks for the thought, unfortunately when running as a plugin Anukari is subject to whatever plist.txt the host application uses. I think that I did try that with the standalone binary at one point, but unfortunately I did not appear to take notes! That probably means I did not have success.
    - aldrich 1y ago
      Very cool work.. and frustating running into walls imposed by manufacturers, I imagine! I've also been working on GPU-based audio plugins for a long time and have done some public material on the subject. Just my two cents: have you considered using a server/daemon process that runs separately and therefore more controllably outside a DAW (and therefore a client-server approach for your plugin instances)? It could allow you to have a little bit more OS-based control.
      - Archit3ch 1y ago
        Do you have a link to your stuff? > have you considered using a server/daemon process that runs separately and therefore more controllably outside a DAW I'm slowly coming to the same conclusion, for audio plugins on GPUs.
- vlovich123 1y ago
  Interesting post & problem. I wonder if the reason that the idea of running the tasks on the same queue fails is for the same reason you have a problem in the first place - variable clock rate means it’s impossible to schedule precisely and you end up aliasing your spin stop time ideal time based on how the OS decided to clock the GPU. But that suggests that maybe your spin job isn’t complex enough to run the GPU at the highest clock because if it is running at max then you should be able to reliably time the stop of the spin even without adding a software PLL (which may not be a bad idea). I didn’t see a detailed explanation of how the spin is implemented and I suspect a more thorough spin loop that consistently drives more of the GPU might be more effective at keeping the clock rate at max perf.
- sunshowers 1y ago
  Great post, I found the description clear and easy to understand. I've definitely run into the issue you're describing in other contexts.
krackers 1y ago
>The Metal profiler has an incredibly useful feature: it allows you to choose the Metal “Performance State” while profiling the application. This is not configurable outside of the profiler. Seems like there might be a private API for this. Maybe it's easier to go the reverse engineering route? Unless it'll end up requiring some special entitlement that you can't bypass without disabling SIP.
- bambax 1y ago
  There has to be a private API for this; the post says: > The Metal profiler has an incredibly useful feature: it allows you to choose the Metal “Performance State” while profiling the application. This is not configurable outside of the profiler. How would the Metal profiler be able to do that if not for a private API? (Could some debugging tool find out what's going on by watching the profiler?)
  - bambax 1y ago
    Lol I just read the parent comment without noticing that they were quoting the exact same sentence from the blog! ;-) Sorry about that!
  - deleted 1y ago
    [deleted]
LiamPowell 1y ago
The problem with exposing an API for this is that far too many developers will force the highest performance state all the time. I don't know if there's really a good way to stop that and have the API at the same time.
- nottorp 1y ago
  Manual permission? Maybe hidden somewhere, it's probably necessary for very niche apps. And default deny at the OS level for Zoom, Teams and web browsers :)
- grishka 1y ago
  There already is an unending number of ways for just one app to waste charge on battery-powered devices. It all already relies on developers not unnecessarily running energy-intensive tasks, either intentionally or accidentally. Adding one more API that has the potential to waste energy if not used appropriately will not change that.
  - madeofpalk 1y ago
    macOS also has a bunch of mechanisms to inform the user about this! IIRC the battery menu has entries for apps draining a lot of power (iterm always shows up there for me!)
    - tonyarkles 1y ago
      My potentially incorrect understanding is that iTerm generally only shows up when the processes you run inside it are consuming a bunch of energy. It only shows up in the battery menu for me when I’m running simulations or other big CPU intensive stuff on the command line.
      - madeofpalk 1y ago
        Yeah - I've always thought about this and was never sure!
- 1y ago
Someone 1y ago
One thing I don’t understand: if latency is important for this use case, why isn’t the CPU busy preparing the next GPU ‘job’ while a GPU ‘job’ is running? Is that a limitation of the audio plug-in APIs?
- grandinj 1y ago
  this might trick the heuristics in the right direction ie. feed the GPU a bunch of small tasks (i.e. with a small number of samples) instead of big tasks.
- kllrnohj 1y ago
  That's pipelining and it's good for throughput but it sacrifices latency. Audio is not a continuous bit stream but a series of small packets. To begin working on the next one on the CPU while the previous one is on the GPU requires 2 samples in flight which necessarily means higher latency
  - Someone 1y ago
    I don’t see that. If the CPU part starts processing packet #2 while the GPU processes packet #1, not after it has done so, it will have the data that has to be sent to the GPU for packet #2 ready earlier, so it can send it earlier, potentially the moment the GPU has finished processing packet #1 (if the GPU is powerful enough, possibly even before that) That’s why I asked about the plug-in APIs. They may have to be async, with functions not returning when they’re fully done processing a ‘packet’ but as soon as they can accept more data, which may be earlier.
    - duped 1y ago
      Audio is already asynchronous. But in general no, you can't begin processing a buffer before finishing the previous buffer because the processing is stateful and you would introduce a data race. And you can't synchronize the state with something simple like a lock, because locking the audio playback is forbidden in real time. You can buffer ahead of time, this introduces latency. You can't do things ahead of time without introducing delay, because of causality - you can't start processing packet #2 while packet #1 is in flight because packet #2 hasn't happened yet. To make it a bit more clear why you can't do this without more latency: Under the hood there is an audio device that reads/writes from a buffer at a fixed interval of time, call that N (number of samples, multiply by sample rate to get in seconds). When that interval is up, the driver swaps the buffer for a new one of the same size. The OS now has exactly (N samples * sample_rate) to fill the buffer before its swapped back with the device driver. The kernel maps or copies the buffer into virtual memory, wake the user space process, call a function to fill the buffer, and return back to kernel space to commit it back to the driver. The buffer you read/write from your process is packet #1. Packet #2 doesn't arrive until the interval ticks again and buffers are exchanged. Now say that processing packet #1 takes longer than N samples or needs at least M samples of data to do its work and M > N. What you do is copy your N samples of packet #1 into a temporary buffer, what until M samples have been acquired to do your work, but concurrently read out of your internal buffer delayed by M - N samples. You've successfully done more work, but delayed the stream by the difference.
threeseed 1y ago
Best way to do this: 1. Go through WWDC videos and find the engineer who seems the most knowledgable about the issue you're facing. 2. Email them directly with this format: mthomson@apple.com for Michael Thomson.
- Hnrobert42 1y ago
  Or his brother Pichael at pthomson.
SOLAR_FIELDS 1y ago
https://xkcd.com/1172/ https://xkcd.com/1172/ feels a lot like the workaround OP describes
- rollcat 1y ago
  That's more like "I had to trick the OS into thinking that spacebar was held for my application to run at all".
sgt 1y ago
I have zero need for this app but it's so cool. Apps like these bring the "fun" back into computing. I don't mean there's no fun at the moment, but reminds me of the old days with more graphical and experimental programs that floated around, even the demoscene.
charcircuit 1y ago
>Any MTLCommandQueue managed by an Audio Workgroup thread could be treated as real-time and the GPU clock could be adjusted accordingly. >The Metal API could simply provide an option on MTLCommandQueue to indicate that it is real-time sensitive, and the clock for the GPU chiplet handling that queue could be adjusted accordingly. Realtime scheduling on a GPU and what the GPU is clocked to are separate concepts. From the article it sounds like the issue is with the clock speeds and not how the work is being scheduled. It sounds like you need something else for providing a hint for requesting a higher GPU clock.
thraway3837 1y ago
This is all just too much Stockholm syndrome. Apple’s DX (developer experience) has always been utterly abysmal, and these continued blog posts just goes to show just how bad it is. Proprietary technologies, poor or no documentation, silent deprecations and removals of APIs, slow trickle feed of yearly WWDC releases that enable just a bit more functionality, introducing newer more entrenched ways to do stuff but still never allowing the basics that every other developer platform has made possible on day 1. A broken UI system that is confusing and quickly becomes undebuggable once you do anything complex. Replaces Autolayout but over a decade of apps have to transition over. Combine framework? Is it dead? Is it alive? Networking APIs that require the use of a 3rd party library because the native APIs don’t even handle the basics easily. Core data a complete mess of a local storage system, still not thread safe. Xcode. The only IDE forced on you by Apple while possibly being the worst rated app on the store. Every update is a nearly 1 hour process of unxipping (yes, .xip) that needs verification and if you skip it, you could potentially have bad actors code inject into your application from within a bad copy of Xcode unbeknownst to you. And it crashes all the time. Swift? Ha. Unused everywhere else but Apple platforms. Swift on server is dead. IBM pulled out over 5 years ago and no one wants to use Swift anywhere but Apple because it’s required. The list goes on. Yet, Apple developers love to be abused by corporate. Ever talk to DTS or their 1-1 WWDC sessions? It’s some of the most condescending, out of touch experience. “You have to use our API this way, and there’s this trick of setting it to this but then change to that and it’ll work. Undocumented but now you know!” Just leave the platform and make it work cross platform. That’s the only way Apple will ever learn that people don’t want to put up with their nonsense.
- eigenspace 1y ago
  It's honestly nuts that so many developers continue to try to make software on MacOS. I understand the appeal of their current hardware, and I used to even be a big fan of the user experience, but it really seems like attempting to build software in MacOS is like trying to build a house on a sandbar. Apple has done nothing and continues to do nothing to engender any confidence in their platform as a development target.
  - gjsman-1000 1y ago
ramesh31 1y ago
Be careful what you wish for here. Knowing Apple, they will stonewall any API requests, and may very well shut your app out for the private API workarounds described.
- mort96 1y ago
  I don't think Anukari is in the Mac App Store, nor do I think a plug-in like it will ever be appropriate for the App Store, so I don't know what exactly you're worried about.
PaulHoule 1y ago
It's an interesting trade-off. For decades the answer to having a reliable Windows computer has been to turn off as many power saving features as possible. Saving power on USB plugs for instance makes your machine crash. Let your CPU state drop to the minimum and you'll find your $3000 desktop computer takes about a second to respond to keypresses. Power savings might not be real, but the crashes and poor performance are very real.
AJRF 1y ago
I’ve worked in two high profile companies with very prominent apps on the Apple App Store. The team we talked to at Apple never ever cared about our problems, but very often invited us to their office to discuss the latest feature they were going to announce at WWDC to strong arm us into supporting it. That was always the start and stop of their engagement with us. We had to burn technical support tickets to ever get any insight into why their buggy software wasn’t working. Apples dev relations are not serious people.
- waffletower 1y ago
  I am glad that your experience is not the rule, as the OP reveals above. However, I worked for a company about 10 years ago with a fairly prominent app. An update that came out that absolutely destroyed the performance of it. At the precisely the same time, a competitor launched an app which did not have the performance difficulty. It turned out that the developer of the competing app had recently left Apple, and left an undocumented surprise in Apple's video drivers that broke it. It took disassembling the competitors binary to find the undocumented change and repair our application. The developer also taunted our CEO by email. Nice world we live in.
  - refulgentis 1y ago
    Wow. I know the thicket of contracts and such makes it not worth it, but I do wish behavior like this could be called out more directly
vessenes 1y ago
Side note: Anukari should put out a Mick Gordon sound pack and share revs with him. That dude is making some crazy crazy stuff; his demo is awesome. Pairing up with artists once you have such a strong tool is good business and good for the world. If you like Mick Gordon. Which I do.
phkahler 1y ago
1024 objects updating at 48khz seems possible on the CPU - depending how the code is written. 48M updates per second? It seems like a possible use for OpenMP to run a few loops in parallel across cores.
- cfstras 1y ago
  If my math is right, that gives you 83 clock cycles to calculate a single sample. on a 16 core, theoretically 1333 cycles. that‘s not a lot, considering you don‘t nearly 100% of the cpu all the time.
- humbledrone 1y ago
  1. Anukari runs up to 16 entire copies of the physics model for polyphony, so 16 * 1024 * 48K (I should update the blog post) 2. Users can arbitrarily connect objects to one another, so each object has to read connections and do processing for N other entities 3. Using the full CPU requires synchronization across cores at each physics step, which is slow 4. Processing per object is relatively large, lots of transcendentals (approx OK) but also just a lot of features, every parameter can be modulated, needs to be NaN-proof, so on 5. Users want to run multiple copies of Anukari in parallel for multiple tracks, effects, etc Another way to look at it is: 4 GHz / (16 voice * 1024 obj * 4 connections * 48,000 sample) = 1.3 cycles per thing The GPU eats this workload alive, it's absolutely perfect for it. All 16 voice * 1024 obj can be done fully in parallel, with trivial synchronization at each step and user-managed L1 cache.
putpoointheloo 1y ago
[flagged]
jonas21 1y ago
I'm having trouble understanding what the problem is -- as in, what are the actual symptoms that users are seeing? How much latency can the app tolerate and how much are you seeing in practice? It would be helpful (to me at least) in thinking about potential solutions if that information were available up front. Perhaps there's something in this video that might help you? They made a lot of changes to scheduling and resource allocation in the M3 generation: https://developer.apple.com/videos/play/tech-talks/111375/ https://developer.apple.com/videos/play/tech-talks/111375/
- humbledrone 1y ago
  It's a real-time audio app, so if it falls behind real time, no audio. You get cracks, pops, and the whole thing becomes unusable. If the user is doing audio at 48 kHz, the required latency is 1/48,000 seconds per sample, or realistically somewhat less than that to account for variance and overhead.
  - lostmsu 1y ago
    I find it hard to believe that users would notice latency under 1ms. Probably not even under 5ms. Have you tried buffering for 5ms? Was result bad? 1 ms?
throwaway48476 1y ago
Another 'appeal to the tsar'?
dgs_sgd 1y ago
> in parallel with the audio computation on the GPU, Anukari runs a second workload on the GPU that is designed to create a high load average and trick macOS into clocking up the GPU. This workload is tuned to use as little of the GPU as possible, while still creating a big enough artificial load to trigger the clock heuristics. That's quite the hack and I feel for the developers. As they state in the post, audio on the GPU is really new and I sadly wouldn't be holding my breath for Apple to cater to it.
notnullorvoid 1y ago
Sorry to hear about the issue, not too surprising given Apples track record with this kind of thing though (You still can't even pin processes to specific CPU core/threads). Anukari is really cool though, wish you had a Linux build :)
philsnow 1y ago
Don't miss the link thrown in the second to last paragraph to https://x.com/Mick_Gordon/status/1918146487948919222 https://x.com/Mick_Gordon/status/1918146487948919222 , a demo Mick Gordon put together, to which @anukarimusic replied > Lol on the second day it's out, you have already absolutely demolished all of the demos I've made with it and I've used it every day for two years
rock_artist 1y ago
While very different, it was already tricky in the past to make Apple silicon (on iPhones as well) perform reasonable. Ableton engineers already evaluated this in the past: https://github.com/Ableton/AudioPerfLab https://github.com/Ableton/AudioPerfLab While I feel for the complaints about the Apple lack of "feedback assiting" The core issue itself is very tricky. Many years ago, before being an audio developer, I've worked in a Pro Audio PC shop... And guess what... interrupts, abusive drivers (GPUs included) and Intels SpeedStep, Sleep states, parking cores... all were tricky. Fast forward, We got asymmetric CPUs, arm64 CPUs and still Intel or AMDs (especially laptops) might need bios tweaks to avoid dropouts/stutters. But if there's a broken driver by CPU or GPU... good luck reporting that one :)
humbledrone 1y ago
Hey everyone, it worked, I had a super productive conversation with exactly the right person on the Metal team! Thanks for helping me get Apple's attention. I didn't at all expect this amount of support. https://anukari.com/blog/devlog/productive-conversation-apple-metal https://anukari.com/blog/devlog/productive-conversation-appl...
- mschuster91 1y ago
  Once again, HN has fulfilled its true purpose: cutting through the red tape that is placed in the front of every large corporation's customer support. Congratulations and good luck with your project!
- krackers 1y ago
  >While I can't share any technical details... The engineer provided some suggestions and hints that I can use right now to maybe — just maybe — get things working in the short term Great that you have a workaround now, but the fact that you can't even share what the workaround is, ironically speaks to the last line in https://news.ycombinator.com/item?id=43904921 https://news.ycombinator.com/item?id=43904921 of how Apple communicates >there’s this trick of setting it to this but then change to that and it’ll work. Undocumented but now you know When you do implement the workaround, maybe you could do it in an overtly-named function spottable via disassembly so that others facing similar constraints of latency-sensitive GPU have some lead as to the magic incantation to use?
  - humbledrone 1y ago
    Despite enjoying your idea, I probably won't do that, but certainly anyone who actually has this same problem should reach out to me and I'll put them in touch with the right folks at Apple, who can share the info.
Liftyee 1y ago
Out of curiosity, what's the origin of the Anukari name?
chrismorgan 1y ago
> (An aside: chalkboards are way better than whiteboards, unless you enjoy getting high on noxious fumes. in which case whiteboards are the way to go.) That looks to be a smoother chalkboard than I’ve ever encountered. If I had been using such chalkboards, I suspect I’d agree, but based purely on my experiences to this point, my opinion has been that chalkboards are significantly better for most art due to finer control and easier and more flexible editing, but whiteboards are better for most teaching purposes (in small or large groups), mostly due to higher contrast. But there’s a lot of variance within both, and placement angles and reflection characteristics matter a lot, as do the specific chalk, markers and ink you use.

405 points by humbledrone 1y ago

ArthurStacks 1y ago

[flagged]

mac9 1y ago
Some of them probably do... This is still a funny comment though

humbledrone 1y ago

Some folks may have seen my Show HN post for Anukari here: https://news.ycombinator.com/item?id=43873074 https://news.ycombinator.com/item?id=43873074 In that thread, the topic of macOS performance came up there. Basically Anukari works great for most people on Apple silicon, including base-model M1 hardware. I've done all my testing on a base M1 and it works wonderfully. The hardware is incredible. But to make it work, I had to implement an unholy abomination of a workaround to get macOS to increase the GPU clock rate for the audio processing to be fast enough. The normal heuristics that macOS uses for the GPU performance state don't understand the weird Anukari workload. Anyway, I finally had time to write down the full situation, in terrible detail, so that I could ask for help getting in touch with the right person at Apple, probably someone who works on the Metal API. Help! :)

aplummer 1y ago
Have you filed a feedback? Seems like the right next step.
- viraptor 1y ago
  Feedback is as effective as creating a change.org petition to some politician to stop doing crimes please. You'll be lucky to get an acknowledgement that something's a real issue after months.
- bayindirh 1y ago
  The post opens with the following TL;DR:, snipped for brevity: > It would be great if someone can connect me with the right person inside Apple, or direct them to my feedback request FB17475838 as well as this devlog entry.
  - sgerenser 1y ago
    Feedbacks often go into a black hole unless either: 1. A bunch of people file effectively the same bug report (unlikely here) 2. An individual Apple employee champions the issue internally 3. Someone makes a fuss on Twitter/X and it starts to go viral Sounds like the OP is trying to get #2 to happen, which is probably his best bet.
    - badc0ffee 1y ago
      Another trick is to schedule some Apple engineer time during WWDC, and plead your case.
      - musicale 1y ago
        I was going to recommend this. They may have some suggestions of how to improve things with existing metal as well.
TheAceOfHearts 1y ago
I missed the Show HN, but the first thing that came to mind after seeing it was that this looks like it would lend itself well to making some very creative ASMR soundscapes with immersive multidimensional audio. I selfishly hope you or one of your users will make a demo. Congrats on the project and I hope you receive help on your Apple issues.
bambax 1y ago
> This is going to be a VERY LONG HIGHLY TECHNICAL post, so either buckle your seatbelt or leave while you still can. Well, I read it all and found it not too long, extremely clear and well-written, and informative! Congrats on the writing. I've never owned a Mac and my pc is old and without a serious GPU, so it's unlikely that I'll get to use Anukari soon, but I regret it very much, as it looks sooo incredibly cool. Hope this gets resolved fast!
Dlemo 1y ago
[flagged]
- mensetmanusman 1y ago
  It’s technical to over half of programmers who don’t need to know these types of details about hw/sw interactions.
  - Dlemo 1y ago
    It's about 'very technical'. If you can explain the problem in one basic sentence it's not very
my123 1y ago
Did you try this entitlement? https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.sustained-execution https://developer.apple.com/documentation/bundleresources/en... wonder if com.apple.developer.sustained-execution also goes the other way around...
- humbledrone 1y ago
  Thanks for the thought, unfortunately when running as a plugin Anukari is subject to whatever plist.txt the host application uses. I think that I did try that with the standalone binary at one point, but unfortunately I did not appear to take notes! That probably means I did not have success.
  - aldrich 1y ago
    Very cool work.. and frustating running into walls imposed by manufacturers, I imagine! I've also been working on GPU-based audio plugins for a long time and have done some public material on the subject. Just my two cents: have you considered using a server/daemon process that runs separately and therefore more controllably outside a DAW (and therefore a client-server approach for your plugin instances)? It could allow you to have a little bit more OS-based control.
    - Archit3ch 1y ago
      Do you have a link to your stuff? > have you considered using a server/daemon process that runs separately and therefore more controllably outside a DAW I'm slowly coming to the same conclusion, for audio plugins on GPUs.
vlovich123 1y ago
Interesting post & problem. I wonder if the reason that the idea of running the tasks on the same queue fails is for the same reason you have a problem in the first place - variable clock rate means it’s impossible to schedule precisely and you end up aliasing your spin stop time ideal time based on how the OS decided to clock the GPU. But that suggests that maybe your spin job isn’t complex enough to run the GPU at the highest clock because if it is running at max then you should be able to reliably time the stop of the spin even without adding a software PLL (which may not be a bad idea). I didn’t see a detailed explanation of how the spin is implemented and I suspect a more thorough spin loop that consistently drives more of the GPU might be more effective at keeping the clock rate at max perf.
sunshowers 1y ago
Great post, I found the description clear and easy to understand. I've definitely run into the issue you're describing in other contexts.

krackers 1y ago

>The Metal profiler has an incredibly useful feature: it allows you to choose the Metal “Performance State” while profiling the application. This is not configurable outside of the profiler. Seems like there might be a private API for this. Maybe it's easier to go the reverse engineering route? Unless it'll end up requiring some special entitlement that you can't bypass without disabling SIP.

bambax 1y ago
There has to be a private API for this; the post says: > The Metal profiler has an incredibly useful feature: it allows you to choose the Metal “Performance State” while profiling the application. This is not configurable outside of the profiler. How would the Metal profiler be able to do that if not for a private API? (Could some debugging tool find out what's going on by watching the profiler?)
- bambax 1y ago
  Lol I just read the parent comment without noticing that they were quoting the exact same sentence from the blog! ;-) Sorry about that!
- deleted 1y ago
  [deleted]

LiamPowell 1y ago

The problem with exposing an API for this is that far too many developers will force the highest performance state all the time. I don't know if there's really a good way to stop that and have the API at the same time.

nottorp 1y ago
Manual permission? Maybe hidden somewhere, it's probably necessary for very niche apps. And default deny at the OS level for Zoom, Teams and web browsers :)
grishka 1y ago
There already is an unending number of ways for just one app to waste charge on battery-powered devices. It all already relies on developers not unnecessarily running energy-intensive tasks, either intentionally or accidentally. Adding one more API that has the potential to waste energy if not used appropriately will not change that.
- madeofpalk 1y ago
  macOS also has a bunch of mechanisms to inform the user about this! IIRC the battery menu has entries for apps draining a lot of power (iterm always shows up there for me!)
  - tonyarkles 1y ago
    My potentially incorrect understanding is that iTerm generally only shows up when the processes you run inside it are consuming a bunch of energy. It only shows up in the battery menu for me when I’m running simulations or other big CPU intensive stuff on the command line.
    - madeofpalk 1y ago
      Yeah - I've always thought about this and was never sure!
1y ago

Someone 1y ago

One thing I don’t understand: if latency is important for this use case, why isn’t the CPU busy preparing the next GPU ‘job’ while a GPU ‘job’ is running? Is that a limitation of the audio plug-in APIs?

grandinj 1y ago
this might trick the heuristics in the right direction ie. feed the GPU a bunch of small tasks (i.e. with a small number of samples) instead of big tasks.
kllrnohj 1y ago
That's pipelining and it's good for throughput but it sacrifices latency. Audio is not a continuous bit stream but a series of small packets. To begin working on the next one on the CPU while the previous one is on the GPU requires 2 samples in flight which necessarily means higher latency
- Someone 1y ago
  I don’t see that. If the CPU part starts processing packet #2 while the GPU processes packet #1, not after it has done so, it will have the data that has to be sent to the GPU for packet #2 ready earlier, so it can send it earlier, potentially the moment the GPU has finished processing packet #1 (if the GPU is powerful enough, possibly even before that) That’s why I asked about the plug-in APIs. They may have to be async, with functions not returning when they’re fully done processing a ‘packet’ but as soon as they can accept more data, which may be earlier.
  - duped 1y ago
    Audio is already asynchronous. But in general no, you can't begin processing a buffer before finishing the previous buffer because the processing is stateful and you would introduce a data race. And you can't synchronize the state with something simple like a lock, because locking the audio playback is forbidden in real time. You can buffer ahead of time, this introduces latency. You can't do things ahead of time without introducing delay, because of causality - you can't start processing packet #2 while packet #1 is in flight because packet #2 hasn't happened yet. To make it a bit more clear why you can't do this without more latency: Under the hood there is an audio device that reads/writes from a buffer at a fixed interval of time, call that N (number of samples, multiply by sample rate to get in seconds). When that interval is up, the driver swaps the buffer for a new one of the same size. The OS now has exactly (N samples * sample_rate) to fill the buffer before its swapped back with the device driver. The kernel maps or copies the buffer into virtual memory, wake the user space process, call a function to fill the buffer, and return back to kernel space to commit it back to the driver. The buffer you read/write from your process is packet #1. Packet #2 doesn't arrive until the interval ticks again and buffers are exchanged. Now say that processing packet #1 takes longer than N samples or needs at least M samples of data to do its work and M > N. What you do is copy your N samples of packet #1 into a temporary buffer, what until M samples have been acquired to do your work, but concurrently read out of your internal buffer delayed by M - N samples. You've successfully done more work, but delayed the stream by the difference.

threeseed 1y ago

Best way to do this: 1. Go through WWDC videos and find the engineer who seems the most knowledgable about the issue you're facing. 2. Email them directly with this format: mthomson@apple.com for Michael Thomson.

Hnrobert42 1y ago
Or his brother Pichael at pthomson.

SOLAR_FIELDS 1y ago

https://xkcd.com/1172/ https://xkcd.com/1172/ feels a lot like the workaround OP describes

rollcat 1y ago
That's more like "I had to trick the OS into thinking that spacebar was held for my application to run at all".

sgt 1y ago

I have zero need for this app but it's so cool. Apps like these bring the "fun" back into computing. I don't mean there's no fun at the moment, but reminds me of the old days with more graphical and experimental programs that floated around, even the demoscene.

charcircuit 1y ago

>Any MTLCommandQueue managed by an Audio Workgroup thread could be treated as real-time and the GPU clock could be adjusted accordingly. >The Metal API could simply provide an option on MTLCommandQueue to indicate that it is real-time sensitive, and the clock for the GPU chiplet handling that queue could be adjusted accordingly. Realtime scheduling on a GPU and what the GPU is clocked to are separate concepts. From the article it sounds like the issue is with the clock speeds and not how the work is being scheduled. It sounds like you need something else for providing a hint for requesting a higher GPU clock.

thraway3837 1y ago

This is all just too much Stockholm syndrome. Apple’s DX (developer experience) has always been utterly abysmal, and these continued blog posts just goes to show just how bad it is. Proprietary technologies, poor or no documentation, silent deprecations and removals of APIs, slow trickle feed of yearly WWDC releases that enable just a bit more functionality, introducing newer more entrenched ways to do stuff but still never allowing the basics that every other developer platform has made possible on day 1. A broken UI system that is confusing and quickly becomes undebuggable once you do anything complex. Replaces Autolayout but over a decade of apps have to transition over. Combine framework? Is it dead? Is it alive? Networking APIs that require the use of a 3rd party library because the native APIs don’t even handle the basics easily. Core data a complete mess of a local storage system, still not thread safe. Xcode. The only IDE forced on you by Apple while possibly being the worst rated app on the store. Every update is a nearly 1 hour process of unxipping (yes, .xip) that needs verification and if you skip it, you could potentially have bad actors code inject into your application from within a bad copy of Xcode unbeknownst to you. And it crashes all the time. Swift? Ha. Unused everywhere else but Apple platforms. Swift on server is dead. IBM pulled out over 5 years ago and no one wants to use Swift anywhere but Apple because it’s required. The list goes on. Yet, Apple developers love to be abused by corporate. Ever talk to DTS or their 1-1 WWDC sessions? It’s some of the most condescending, out of touch experience. “You have to use our API this way, and there’s this trick of setting it to this but then change to that and it’ll work. Undocumented but now you know!” Just leave the platform and make it work cross platform. That’s the only way Apple will ever learn that people don’t want to put up with their nonsense.

eigenspace 1y ago
It's honestly nuts that so many developers continue to try to make software on MacOS. I understand the appeal of their current hardware, and I used to even be a big fan of the user experience, but it really seems like attempting to build software in MacOS is like trying to build a house on a sandbar. Apple has done nothing and continues to do nothing to engender any confidence in their platform as a development target.
- gjsman-1000 1y ago

ramesh31 1y ago

Be careful what you wish for here. Knowing Apple, they will stonewall any API requests, and may very well shut your app out for the private API workarounds described.

mort96 1y ago
I don't think Anukari is in the Mac App Store, nor do I think a plug-in like it will ever be appropriate for the App Store, so I don't know what exactly you're worried about.

PaulHoule 1y ago

It's an interesting trade-off. For decades the answer to having a reliable Windows computer has been to turn off as many power saving features as possible. Saving power on USB plugs for instance makes your machine crash. Let your CPU state drop to the minimum and you'll find your $3000 desktop computer takes about a second to respond to keypresses. Power savings might not be real, but the crashes and poor performance are very real.

AJRF 1y ago

I’ve worked in two high profile companies with very prominent apps on the Apple App Store. The team we talked to at Apple never ever cared about our problems, but very often invited us to their office to discuss the latest feature they were going to announce at WWDC to strong arm us into supporting it. That was always the start and stop of their engagement with us. We had to burn technical support tickets to ever get any insight into why their buggy software wasn’t working. Apples dev relations are not serious people.

waffletower 1y ago
I am glad that your experience is not the rule, as the OP reveals above. However, I worked for a company about 10 years ago with a fairly prominent app. An update that came out that absolutely destroyed the performance of it. At the precisely the same time, a competitor launched an app which did not have the performance difficulty. It turned out that the developer of the competing app had recently left Apple, and left an undocumented surprise in Apple's video drivers that broke it. It took disassembling the competitors binary to find the undocumented change and repair our application. The developer also taunted our CEO by email. Nice world we live in.
- refulgentis 1y ago
  Wow. I know the thicket of contracts and such makes it not worth it, but I do wish behavior like this could be called out more directly

vessenes 1y ago

Side note: Anukari should put out a Mick Gordon sound pack and share revs with him. That dude is making some crazy crazy stuff; his demo is awesome. Pairing up with artists once you have such a strong tool is good business and good for the world. If you like Mick Gordon. Which I do.

phkahler 1y ago

1024 objects updating at 48khz seems possible on the CPU - depending how the code is written. 48M updates per second? It seems like a possible use for OpenMP to run a few loops in parallel across cores.

cfstras 1y ago
If my math is right, that gives you 83 clock cycles to calculate a single sample. on a 16 core, theoretically 1333 cycles. that‘s not a lot, considering you don‘t nearly 100% of the cpu all the time.
humbledrone 1y ago
1. Anukari runs up to 16 entire copies of the physics model for polyphony, so 16 * 1024 * 48K (I should update the blog post) 2. Users can arbitrarily connect objects to one another, so each object has to read connections and do processing for N other entities 3. Using the full CPU requires synchronization across cores at each physics step, which is slow 4. Processing per object is relatively large, lots of transcendentals (approx OK) but also just a lot of features, every parameter can be modulated, needs to be NaN-proof, so on 5. Users want to run multiple copies of Anukari in parallel for multiple tracks, effects, etc Another way to look at it is: 4 GHz / (16 voice * 1024 obj * 4 connections * 48,000 sample) = 1.3 cycles per thing The GPU eats this workload alive, it's absolutely perfect for it. All 16 voice * 1024 obj can be done fully in parallel, with trivial synchronization at each step and user-managed L1 cache.

putpoointheloo 1y ago

[flagged]

jonas21 1y ago

I'm having trouble understanding what the problem is -- as in, what are the actual symptoms that users are seeing? How much latency can the app tolerate and how much are you seeing in practice? It would be helpful (to me at least) in thinking about potential solutions if that information were available up front. Perhaps there's something in this video that might help you? They made a lot of changes to scheduling and resource allocation in the M3 generation: https://developer.apple.com/videos/play/tech-talks/111375/ https://developer.apple.com/videos/play/tech-talks/111375/

humbledrone 1y ago
It's a real-time audio app, so if it falls behind real time, no audio. You get cracks, pops, and the whole thing becomes unusable. If the user is doing audio at 48 kHz, the required latency is 1/48,000 seconds per sample, or realistically somewhat less than that to account for variance and overhead.
- lostmsu 1y ago
  I find it hard to believe that users would notice latency under 1ms. Probably not even under 5ms. Have you tried buffering for 5ms? Was result bad? 1 ms?

throwaway48476 1y ago

Another 'appeal to the tsar'?

dgs_sgd 1y ago

> in parallel with the audio computation on the GPU, Anukari runs a second workload on the GPU that is designed to create a high load average and trick macOS into clocking up the GPU. This workload is tuned to use as little of the GPU as possible, while still creating a big enough artificial load to trigger the clock heuristics. That's quite the hack and I feel for the developers. As they state in the post, audio on the GPU is really new and I sadly wouldn't be holding my breath for Apple to cater to it.

notnullorvoid 1y ago

Sorry to hear about the issue, not too surprising given Apples track record with this kind of thing though (You still can't even pin processes to specific CPU core/threads). Anukari is really cool though, wish you had a Linux build :)

philsnow 1y ago

Don't miss the link thrown in the second to last paragraph to https://x.com/Mick_Gordon/status/1918146487948919222 https://x.com/Mick_Gordon/status/1918146487948919222 , a demo Mick Gordon put together, to which @anukarimusic replied > Lol on the second day it's out, you have already absolutely demolished all of the demos I've made with it and I've used it every day for two years

rock_artist 1y ago

While very different, it was already tricky in the past to make Apple silicon (on iPhones as well) perform reasonable. Ableton engineers already evaluated this in the past: https://github.com/Ableton/AudioPerfLab https://github.com/Ableton/AudioPerfLab While I feel for the complaints about the Apple lack of "feedback assiting" The core issue itself is very tricky. Many years ago, before being an audio developer, I've worked in a Pro Audio PC shop... And guess what... interrupts, abusive drivers (GPUs included) and Intels SpeedStep, Sleep states, parking cores... all were tricky. Fast forward, We got asymmetric CPUs, arm64 CPUs and still Intel or AMDs (especially laptops) might need bios tweaks to avoid dropouts/stutters. But if there's a broken driver by CPU or GPU... good luck reporting that one :)

humbledrone 1y ago

Hey everyone, it worked, I had a super productive conversation with exactly the right person on the Metal team! Thanks for helping me get Apple's attention. I didn't at all expect this amount of support. https://anukari.com/blog/devlog/productive-conversation-apple-metal https://anukari.com/blog/devlog/productive-conversation-appl...

mschuster91 1y ago
Once again, HN has fulfilled its true purpose: cutting through the red tape that is placed in the front of every large corporation's customer support. Congratulations and good luck with your project!
krackers 1y ago
>While I can't share any technical details... The engineer provided some suggestions and hints that I can use right now to maybe — just maybe — get things working in the short term Great that you have a workaround now, but the fact that you can't even share what the workaround is, ironically speaks to the last line in https://news.ycombinator.com/item?id=43904921 https://news.ycombinator.com/item?id=43904921 of how Apple communicates >there’s this trick of setting it to this but then change to that and it’ll work. Undocumented but now you know When you do implement the workaround, maybe you could do it in an overtly-named function spottable via disassembly so that others facing similar constraints of latency-sensitive GPU have some lead as to the magic incantation to use?
- humbledrone 1y ago
  Despite enjoying your idea, I probably won't do that, but certainly anyone who actually has this same problem should reach out to me and I'll put them in touch with the right folks at Apple, who can share the info.

Liftyee 1y ago

Out of curiosity, what's the origin of the Anukari name?

chrismorgan 1y ago

> (An aside: chalkboards are way better than whiteboards, unless you enjoy getting high on noxious fumes. in which case whiteboards are the way to go.) That looks to be a smoother chalkboard than I’ve ever encountered. If I had been using such chalkboards, I suspect I’d agree, but based purely on my experiences to this point, my opinion has been that chalkboards are significantly better for most art due to finer control and easier and more flexible editing, but whiteboards are better for most teaching purposes (in small or large groups), mostly due to higher contrast. But there’s a lot of variance within both, and placement angles and reflection characteristics matter a lot, as do the specific chalk, markers and ink you use.