7 ms·
It seems to be an import pipeline bug. Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional dele
by deviation 1y ago
It seems to be an import pipeline bug.
Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional deletion), so my guess is a concurrency bug where a buffer gets reused or a file handle is closed before the copy finishes.
Rare, nondeterministic corruption fits the profile.
- BrtByte 1y agoWhat’s wild is that this kind of bug feels like something that should’ve been caught with even minimal end-to-end integrity checks
- jonny_eh 1y agoTests can be flaky, and then ignored
- tenderlove 1y agoThis is also my guess. It's really a bummer, and I'd report it to Apple but since it's nondeterministic I have no idea how to provide repro steps.
- ChrisMarshallNY 1y agoI have had extremely bad luck, reporting bugs to Apple. They constantly ask for an example project, even if it's something that is easily demonstrated, simply by running existing Apple software, and creating a project, would be a huge pain. They also ignore reports. Very rarely, I may get a ping on one of my reports, asking me to verify that it was fixed in some release. Otherwise, there's no sign that they ever even read it. I usually end up closing my bug reports and feature requests, after a few months, because I'm tired of looking at them. It's clear that they consider every bug report to be a burden. That's a very strange stance, but then, they are not a typical company. I guess you can't argue with the results, as they have a market value North of 3 trillion dollars, but that does not make it any less annoying.
- deviation 1y agoNot to hand wave-- but this feels industry standard IMO. I have a dozen PRs sitting unacknowledged and stale across a handful of FAANG (and other) repos, including Apple's. I start my first day @ Apple in a few weeks, so I ACK that my opinion might be a little biased here.
- dmd 1y agoMaybe you can help bump FB13400242, a bug that is _literally_ going to kill people. (The bug is that to make an emergency call, even from lock screen, you're supposed to be able to squeeze buttons on either side of the phone. But it only works with the volume buttons on the left - the Action button didn't get supported, when that button was added. So now the rule for teaching a small child isn't just "squeeze both sides" it's "oh but not that one!") (Yes, this came close to killing someone close to me. Fortunately someone else happened to come along to help.)
- RankingMember 1y agoConsider hitting up some Apple "watcher" people (e.g. 9to5mac) to see if they can give you a boost on their social media. It's pretty obnoxious that it's come to needing to make a stink like that to get eyeballs on something, but here we are.
- strunz 1y agoHave you tried copying the files to the local disk before importing?
- turnsout 1y agoThis is what I always do. Rather than go directly from the card reader or camera into Photos or Lightroom, I copy the files onto an SSD, and then bring them in from the SSD. The entire process goes faster. I also want to point out that I've seen similar corruption in the past, only in Lightroom. The culprit ended up being hardware, not software. Specifically, the card reader's USB cable. I've actually had two of these cables fail on different readers. On the most recent one, I replaced it with a nicer Micro B to USB C cable, and haven't had an issue.
- PaulHoule 1y agoI haven't had actual corruption but had imports take an excessive long time or fail to complete in Lightroom because of bad USB cables or (I think) bad USB jack. Generally I'm frustrated with the state of USB. Bad cables are all over the place and I'm inclined to throw cables out if I have the slightest problem with them. My take is that the import process with Lightroom is fast and reliable if I am using good readers and good cables; it is fine importing photos from my Sony a7iv off a CFExpress card but my Sony a7ii has always been problematic and benefits greatly from taking the memory card out and putting it in a dedicated reader, sometimes I use the second slot in the a7iv.
- sib 1y agoI use Lightroom, but always with this workflow (copy files from memory card to disk, then use LR to do the import / move / build previews). If nothing else, it lets you get your card back much more quickly, as a file-system copy runs at ~1500MBps, which makes a difference when importing 50-100GB of photos. I also don't delete the images off the memory card until they've been backed up from the disk to some additional medium.
- vladvasiliu 1y ago
- ValentineC 1y agoWhat's worse about the Apple ecosystem is that because of how tightly coupled it is, a bug fix for Photos would only come as part of a macOS update. Which means that if that bug has been present since the (now unsupported) Mavericks, tough luck!
- JKCalhoun 1y agoMacOS updates are several per year though. If a fixes found (and the bug considered a high enough priority) it could show up before 2025 is up.
- conductr 1y agoIt’s a way to force you to update those apps. They don’t want you on the latest OS and on an old Music or Photos.
- ValentineC 1y agoSure, but I want to be on an older macOS and be given the option to use the latest Music or Photos. For example, I'm probably not going to be upgrading to Tahoe until Sequoia's close to end of support life. I just upgraded to Sequoia last month from Ventura (decided to skip Sonoma since the various anecdotes of Sequoia made it sound like Sonoma-with-AI, with few apps breaking because of some new API change).
- Gigachad 1y agoIt’s more a problem for Macs not supported with OS updates like the 2019 13” models right now.
- thewebguyd 1y agoI still don't under stand why Apple limits updates to their first party apps to OS updates. They could really benefit from how Google does it on Android and decouple it. Push updates to their first party apps via the app store like everyone else, and let the OS update on its own separate schedule.
- egorfine 1y ago> I'd report it to Apple What's the point of it? It is well known in the industry they ignore bugreports. Also, this bug doesn't affect the majority of users, so it won't ever be fixed.
- JKCalhoun 1y agoI worked on the Photos team a decade ago — some of what you're saying I can vouch for. If it is a rare occurrence, that lowers the priority of the bug. Data corruption though? That moves it to the top. I'll tell you a secret though that kind of pisses me off. If you have shipped with a bug, that automatically lowers the perceived priority as well. You know, as opposed to introducing a new bug in a new release. "We've already lived with that old bug…" seems to be the mind set. Oh well. To be sure though, if you saw the number of bugs that queue up for a popular app like Photos, you'd know that fixing all of them is not going to be possible — some kind of system of prioritization is required.
- ryandrake 1y ago> I'll tell you a secret though that kind of pisses me off. If you have shipped with a bug, that automatically lowers the perceived priority as well. You know, as opposed to introducing a new bug in a new release. This mentality is all over BigTech: This bug didn't block release X-1, why should it block release X? So, it inevitably just sits in the backlog forever. If your releases are 90 days apart, any bug found has an average of 45 days to be fixed, or it ends up on the "we lived with it last time" list.
- throwaway31131 1y agoIf you have more bugs than you can fix in a given amount of time then you have to prioritize somehow. “This bug didn't block release X-1, why should it block release X?” Is actually a pretty strong argument and tough, but not impossible, to counter. And the bug backlog only gets longer with time. It’s the price of greatly increased software complexity.
- egorfine 1y ago
- runjake 1y agoI think your best bet is to do what you did: write an extensive blog post about it and hope it goes "viral" and grabs the right people at Apple's attention. I would think the diffs would be telling to the right people. It's on the front page of HN, so that's a good start!
- JKCalhoun 1y agoFront page of HN is all it takes. Pretty sure it's viral.
- deleted 1y ago[deleted]
- groos 1y agoYou don't need repro steps if Apple is serious about quality. Just the description of what's happening should give enough to a senior Apple engineer to intuit where this is possibly happening and create tests that will stress their software to repro this.
- AceJohnny2 1y agoI see you've never used Feedback Assistant
- JKCalhoun 1y agoYou're right. I too have been frustrated when spending the effort to file a bug only to have it "auto-returned" with "cannot repro, please attach example project" or somesuch. As an engineer on a framework, I take pride in the code and want those trickier-to-repro bugs. And to your point, yeah, I can probably reconstruct in my mind a scenario, knowing my code, where it may be vulnerable. Whoever screens bugs though is likely trying to 1) handle the huge backlog and 2) trying to spare the engineers bugs delivered with a shrug. (I had my ass kicked by one bug in particular where I created a FileDescriptor on the main thread, did some things and then released the FileDescriptor on a background thread. Who knew the FileDescriptor code hung some important data it needed on some kind of thread-based storage that essentially required the caller to destroy the FileDescriptor on the same thread it was created on. Fix was to dispatch back to main when ready to release the FileDescriptor.)
- girvo 1y agoiTunes Match has routinely just... removed/greyed out files that I've had uploaded, and I've reported this to Apple over and over again, with exactly zero explanation of why or how to avoid it. To this day it still does it.
- inanutshellus 1y agoInteresting that he went from 30% failure to it taking a while to find a single failure after replacing everything. Random is random, and random is clumpy, so maybe swapping parts is irrelevant, but... I wanted more detail how often the corruption happened throughout his replacement journey. edit: also wth i just realized I went to "tenderlovemaking.com" at work. gross. lol.
- bluSCALE4 1y agoI'd be interested in knowing if he was multitasking and using a lot of memory. I know wedding photos are usually something you feel rushed to upload so maybe this issue can be made worse depending on system resource availability.
- inanutshellus 1y agoyeah copying can take a loooooooong time and so you multitask. maybe the randomness is based on the other apps he's using at the same time.
- tuetuopay 1y agoSince it looks like a concurrency issue, most likely the new laptop made the issue less frequent through the simple virtue of being faster.
- eviks 1y agoThat's not everything that happened, a big non-replacement part > I stopped checking the “delete after import” button
- deleted 1y ago[deleted]
- JoBrad 1y agoI wonder if it’s related to import sources, and maybe the speed of that hardware. They are still successfully importing the photos into the Photos app, just not from the camera.
- mentos 1y agoSounds like an argument for Apple to provide a new high-level media import framework?
- Someone 1y agoWhy? A new framework gets you new bugs instead of the old ones, but not necessarily fewer or less severe ones. It’s more likely that things will be reversed: the old, battle-tested framework may have bugs, but it’s is less likely to have serious ones. They should try to hunt down bugs in the existing code. A partial rewrite of parts that historically have many bugs may be in order, but a complete replacement? Unlikely to be an improvement.
- mentos 1y agoWell I'd imagine the new framework wouldn't rewrite old but wrap the existing low level APIs in a way that is not error prone. Centralize the tricky bits so Photos and third party apps don’t each have to reinvent them?
- loeg 1y agoMaybe bad RAM flipping bits? Edit: Nevermind, the contents are vastly changed. This is like a different stream of input got used, or a buffer was written over with contents from another image.
- aakkaakk 1y agoThis bug has happened on multiple, perhaps all, iphones I ever owned, I clearly remember at least 3 where I spent hours on trying to save (extract) photos from my iphone with different tools. ”Glad” to see it was an actual bug.
- sixothree 1y agoI have a few corrupted photos in iCloud that were taken with my iPhone. Maybe there's something larger going on here.