10 ms·
For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wro
by rishabhjain1198 2y ago
For people interested in AI research, there's nothing new here.
IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development".
Benchmarks are nice though.
- kfrzcode 2y ago"AI for the rest of us."
- wkat4242 2y agoExcept Apple isn't really for the rest of us. Outside of America and a handful wealthy western countries it's for the top 5-20% earners only.
- throwaway2037 2y agoJapan and Taiwan are both more than 50% iOS. Ref: https://worldpopulationreview.com/country-rankings/iphone-market-share-by-country https://worldpopulationreview.com/country-rankings/iphone-ma...
- jahewson 2y agoApproximately 33% of all smartphones in the world are iPhones.
- theshrike79 2y agoIn the EU the market share is 30%
- d1sxeyes 2y agoYes but not evenly distributed, BeNeLux, Germany, Austria, and Nordic countries have a lot of iPhone users, while moving further east (or south) you see lower market share. Maybe it’s “two handfuls” of wealthy western countries rather than just one, but I think OPs point holds true.
- kolinko 2y agoIn Poland it’s 33%
- elbear 2y agoIn Romania it's 24.7%
- d1sxeyes 2y agoHuh interesting, I missed that. You’re right (actually I see even 25.5%).
- theshrike79 2y agohttps://worldpopulationreview.com/country-rankings/iphone-market-share-by-country https://worldpopulationreview.com/country-rankings/iphone-ma... Poland, Greece, Hungary and Bosnia-Herzegovina are the only ones under 20% (and maybe a few others). OTOH Britain is over 50% as is Sweden. Finland, the land of Nokia is over 35%.
- whynotminot 2y agoWho do you think this presentation is geared toward?
- chuckjchen 2y agoThis sounds like every newcomers to the stage except for big players like Apple.
- marcellus23 2y agoThey refer to LoRA explicitly in the post.
- rishabhjain1198 2y agoAlthough I caught that on the first read, I found myself questioning when I read the adaptors part, "is this not just LoRA...". Maybe it's my fault as a reader, but I think the writing could be clearer. Usually in a research paper you would link to the LoRA paper there too.
- WiSaGaN 2y agoThis gives me the vibe of calling high resolution screens as "retina" screens.
- dishsoap 2y agoI don't see anything wrong with that at all. They've created a branding term that allows consumers to get an idea of the sort of pixel density they can expect without having to actually check, should they not want to bother.
- necovek 2y agoExcept that everyone has different visual acuity and different distance they use the same devices at, and in the end, "retina" means nothing at all. But this is exactly the type of marketing Apple is good at, though "retina" is probably not the most successful example.
- theshrike79 2y agoIf your "visual acuity" is so good that you can see the pixels of a retina-branded display from the intended viewing distance, you might need to be studied for science.
- jackothy 2y agoIt's not so impossible to spot flaws if you're using worst-case testing scenarios. Which are not worthless because such patterns do actually pop up in real world usage, albeit rarely.
- kolinko 2y agoExamples?
- jackothy 2y agoHad one happen to me recently where I was scrolling Spotify, and they do the thing where if you try to scroll past max they will stretch the content. One of the album covers being stretched had some kind of fine pattern on it that caused a clearly visible shifting/flashing Moiré pattern as it was being stretched. Wish I could remember what album cover it was now. Though really it's simple enough: As long as you can still spot a single dark pixel in the middle of an illuminated white screen, the pixels could benefit from being smaller. (Edit: swapped black and white)
- threeseed 2y agoIt's a huge development in terms of it being a consumer-ready, on-device LLM. And if Karpathy thinks so then I assume it's good enough for HN: https://x.com/karpathy/status/1800242310116262150 https://x.com/karpathy/status/1800242310116262150
- rishabhjain1198 2y agoThe productization of it (like Karpathy mentioned) is awesome. But I think the URL for that would be this maybe? [link](https://www.apple.com/apple-intelligence/ https://www.apple.com/apple-intelligence/)
- kfrzcode 2y ago[flagged]
- threeseed 2y agoa) I would trust Karpathy over Elon given he doesn't have a competing product. b) Apple only provides information to ChatGPT when the user consents to doing so and the information is only for that request i.e. it is not logged for future training.
- pests 2y agoThe temp around Elon here is lower than you think. I would say almost the exact opposite of your claim.
- slimebot80 2y agoElon is talking out his arse as usual.
- camillomiller 2y agoHe is factually wrong and has been rekted by his own community notes
- gigglesupstairs 2y agoWas there anything about searching through our own photos using prompts? I thought this could be pretty amazing and still a natural way to find very specific photos in one’s own photo gallery.
- avereveard 2y agoWhich is in turn just multimodal embedding Besides I could do "named person on a beach in August" and get the correct thing in photos on Android photos, so I don't get it. It's amazing for apple users if they didn't have it before. But from a tech stand point people could have had it for a while.
- azinman2 2y agoPhotos has had this for a while with structured natural language queries, and this kind of prompt was part of the WWDC video.
- theshrike79 2y agoThe difference is that Apple has been doing this on-device for maybe 4-5 years already with the Neural Engine. Every iOS version has brought more stuff you can search for. The current addition is "just" about adding a natural language interface on top of data they already have about your photos (on device, not in the cloud). My iPhone 14 can, for example, detect the breed of my dog correctly from the pictures and it can search for a specific pet by name. Again on-device, not by sending my stuff to Google's cloud to be analysed.
- fauigerzigerk 2y agoThey have been trying and failing to do a tiny little bit of this. It's so broken and useless that I've been uploading all my iCloud photos to Google as well, for search and sharing.
- theshrike79 2y agoIf you like Google using your personal photos for machine learning, that's your option. Now they have your every photo, geotagged and timestamped so they can see where you have been and at what times. Then they of course anonymise that information into an "advertiser id" they tag on to you and a sufficient quantity of other people so they can claim they're not directly targeting anyone. I prefer Apple's privacy focused option myself.
- lolinder 2y ago> For people interested in AI research, there's nothing new here. Was anyone expecting anything new? Apple has never been big on living at the cutting edge of technology exploring spaces that no one has explored before—from laptops to the iPhone to iPads to watches, every success they've had has come from taking tech that was already prototyped by many other companies and smoothing out the usability kinks to get it ready for the mainstream. Why would deep learning be different?
- deleted 2y ago[deleted]
- jeanlucas 2y ago> For people interested in AI research I think he is pointing out for people interested in research. OTOH, it is interesting to see how a company is applying AI to customers at the end. It will bring up new challenges that will be interesting from at least an engineering point of view.
- IOT_Apprentice 2y agoApple was first with 64 bit iPhone chips. Remember Qualcomm VP at the time claimed it was nothing. Apple Silicon for M1 was impressive for instant in low power high performance.
- lolinder 2y agoThose are both still (major) incremental improvements to known tech, not cutting-edge research. Apple takes what other companies have already done and does it better.
- sunshinerag 2y agoall cutting-edge research other companies are supposedly doing are also incremental. Depends on your vantage point.
- prmoustache 2y agoBut last at bringing a calculator on the iPad =)
- derefr 2y agoI think the thing they're saying that's novel, isn't what they have (LoRAs), but where and when and how they make them. Rather than just pre-baking static LoRAs to ship with the base model (e.g. one global "rewrite this in a friendly style" LoRA, etc), Apple seem to have chosen a bounded set of behaviors they want to implement as LoRAs — one for each "mode" they want their base model to operate in — and then set up a pipeline where each LoRA gets fine-tuned per user, and re-fine-tuned any time the data dependencies that go into the training dataset for the given LoRA (e.g. mail, contacts, browsing history, photos, etc) would change. In other words, Apple are using their LoRAs as the state-keepers for what will end up feeling to the user like semi-online Direct Preference Optimization. (Compare/contrast: what Character.AI does with their chatbot response ratings.) --- I'm not as sure, from what they've said here, whether they're also implying that these models are being trained in the background on-device. It could very well be possible: training something that's only LoRA-sized, on a vertically-integrated platform optimized for low-energy ML, that sits around awake but doing nothing for 8 hours a day, might be practical. (Normally it'd require a non-quantized copy of the model, though. Maybe they'll waste even more of your iPhone's disk space by having both quantized and non-quantized copies of the model, one for fast inference and the other for dog-slow training?) But I'm guessing they've chosen not to do this — as, even if it were practical, it would mean that any cloud-offloaded queries wouldn't have access to these models. Instead, I'm guessing the LoRA training is triggered by the iCloud servers noticing you've pushed new data to them, and throwing a lifecycle notification into a message queue of which the LoRA training system is a consumer. The training system reduces over changes to bake out a new version of any affected training datasets; bakes out new LoRAs; and then basically dumps the resulting tensor files out into your iCloud Drive, where they end up synced to all your devices.
- wmf 2y agoI don't think the LoRAs are fine-tuned locally at all. It sounds like they use RAG to access data.
- derefr 2y agoConsider a feature from earlier in the keynote: the thing Notes (and Math Notes) does now where it fixes up your handwriting into a facsimile of your handwriting, with the resulting letters then acting semantically as text (snapping to a baseline grid; being reflowable; being interpretable as math equations) but still having the kind of long-distance context-dependent variations that can't be accomplished by just generating a "handwriting font" with glyph variations selected by ligature. They didn't say that this is an "AI thing", but I can't honestly see how else you'd do it other than by fine-tuning a vision model on the user's own handwriting.
- rvaish 2y agoreminds me of Easel on iMessage: https://easelapps.ai/ https://easelapps.ai/
- throwaway4good 2y agoI thought the news of them using Apple Silicon rather than NVIDIA in their data centers was significant. Perhaps there is still hope of a relaunch of xserve; with the widespread use of Apple computers amongst developers Apple has a real chance of challenging NVIDIA's CUDA moat.
- pjmlp 2y agoNot at Apple's price points.
- throwaway4good 2y agoI think NVIDIA has the highest hardware markup at the moment.
- pjmlp 2y agoDepends on which card one is talking about.
- throwaway4good 2y agoMaybe. It is not really obvious how much you for the AI accellerator part of their offerings. For example the chips in iPhones are quite powerful even adjusted for price. However for some cases - like the max chip in the macbooks or the extra ram - their pricing seems high - maybe even nvidia high.
- bayindirh 2y ago[flagged]
- pjmlp 2y agoYes, it does. Should we keep arguing like on school playground?
- 2y ago
- Cthulhu_ 2y agoThing is, Apple takes these concepts and polishes them, makes them accessible to maybe not laypeople but definitely a much wider audience compared to those already "in the industry", so to speak.
- jan3024 2y ago[dead]
- franzb 2y agoThis isn't about AI research, it's about delivering AI at unimaginable scale.
- throwthrowuknow 2y ago180 million users for chatgpt isn’t unimaginable but it does exceed the number of iPhone users in the United States.
- scosman 2y agoI think you’re referring to my comment about this being huge for developers? Just want to point out I call this launch huge, didn’t say “huge development” as quoted, and didn’t imply what was interesting was the ML research. No one in this thread used the quoted words, at least that I can see. My comment was about dev experience, memory swapping, potential for tuning base models to each HW release, fine tune deployment, and app size. Those things do have the potential to be huge for developers, as mentioned. They are the things that will make a local+private ML developer ecosystem work. I think the article and comment make sense in their context: a developer conference for Mac and iOS devs. Apple also explicitly says it’s LoRA.
- avidphantasm 2y agoVery little of the “AI” boom has been novel, most has been iterative elaborations (though innovative nonetheless). Academics have been using neural network statistical models for decades. What’s new is the combination of compute capability and data volume available for training. It’s iterative all the way down though, that’s how all technologies are developed.
- sigmoid10 2y agoMost people don't realize this, but almost all research works that way. Only the media spins research as breakthrough-based, because that way it is easier to sell stories. But almost everything is incremental/iterative. Even the transformer architecture, which in some way can be seen as the most significant architectural advancement in AI in the past years, was a pretty small, incremental step when it came out. Only with a lot of further work building on top of that did it become what we see today. The problem is that science-journalists vastly outnumber scientists producing these incremental steps, so instead of reporting on topics when improvements actually accumulated to a big advancement, every step along the way gets its own article with tons of unnecessary commentary heralding its features.
- w10-1 2y ago> What’s new is the combination of compute capability and data volume available for training This is the important part. My advisor said new means old method applied to new data or new method on old data. Commercially, that means price points, i.e., discrete points where something becomes viable. Maybe that's iterative, but maybe not. Either way, once the opportunity presents, time is of the essence.
- astrange 2y agoThe "bitter lesson of machine learning" means that you actually can't do anything novel; it won't work as well as just doing the simple thing but bigger. (So there is room left if you're limited by memory or budget.)
- steve1977 2y agoYou know what company you are talking about here?
- lhl 2y agoI think your conclusion is uncharitable or at least depends on how deep your interest in AI research actually is. Reading the docs, there are at least several points of novelty/interest: * Clearly outlining their intent/policies for training/data use. Committing to no using user data or interactions for training their base models is IMO actually a pretty big deal and a differentiator from everyone else. * There's a never-ending stream of new RL variants ofc, but that's how technology advances, and I'm pretty interested to see how these compare with the rest: "We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator. We find that these two algorithms lead to significant improvement in the model’s instruction-following quality." * I'm interested to see how their custom quantization compares with the current SoTA (probably AQLM atm) * It looks like they've done some interesting optimizations to lower TTFT, this includes the use of some sort of self-speculation. It looks like they also have a new KV-cache update mechanism and looking forward to reading about that as well. 0.6ms/token means that for your average I dunno, 20 token query you might only wait 12ms for TTFT (I have my doubts, maybe they're getting their numbers from much larger prompts, again, I'm interested to see for myself) * Yes, it looks like they're using pretty standard LoRAs, the more interesting part is their (automated) training/re-training infrastructure but I doubt that's something that will be shared. The actual training pipeline (feedback collection, refinement, automated deployment) is where the real meat and potatoes of being able to deploy AI for prod/at scale lies. Still, what they shared about their tuning procedures is still pretty interesting, as well as seeing which models they're comparing against. As this article doesn't claim to be a technical report or a paper, while citations would be nice, I can also understand why they were elided. OpenAI has done the same (and sometimes gotten heat for it, like w/ Matroyshka embeddings). For all we know, maybe the original author had references, or maybe since PEFT isn't new to those in the field, that describing it is just being done as a service to the reader - at the end of the day, it's up to the reader to make their own judgements on what's new or not, or a huge development or not. From my reading of the article, your conclusion, which funnily enough is now the new top-rated comment on this thread isn't actually much more accurate the the one old one you're criticizing.
- monkeydust 2y agoFeel Apple should have just focused on their models for this one and not complicate the conversation with OpenAI. They could have left that to another announcement later. Quick straw poll survey around the office, many think their data will be sent off to OpenAI by default for these new features which is not the case.
- Spooky23 2y agoThose people aren’t looking at Apple. They seem to have a good model for adding value to their products without the hold my beer, conquer the world bullshit that you get from OpenAI, et al.
- deleted 2y ago[deleted]
- frompom 2y agoDo you have the same expectations for any company launching hardware that they cite the various papers related to how the tech was developed? EVERY piece of tech announced by ANY company relies on a variety of research out there yet it doesn't seem expected that every time anyone launches something they cite the numerous papers related to that. Why would products/services in this category be any different?
- talldayo 2y ago> Do you have the same expectations for any company launching hardware that they cite the various papers related to how the tech was developed? If they try to market it with a seemingly unique or yet-unheard of name, then yeah. It is nice knowing what the "real world" name of an Apple-ized technology is. Just ignoring it and marketing the technology under some new name is adjacent to lying to your audience through omission.
- dwaite 2y ago> Just ignoring it and marketing the technology under some new name is adjacent to lying to your audience through omission. They don't market technology, they market solutions. E.g. afib detection on Apple Watch, rather than calling it a BNNS using a custom-built sensor for one-wire EKG. This is the document where they describe how the solution works, and they clearly state adapters work based on LoRA.
- arvinsim 2y ago> The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. That's a classic Apple strategy though.