6 ms·

Translating an ARM iOS App to Intel macOS Using Bitcode

148 points by sjuut 7y ago

sjtgraham 7y ago
This is rather "embed intermediate representation, extract, run another target's back end, and link". I was hoping this would be explaining how to go from ARM by uplifting executable to LLVM IR and retargeting for Intel. This is notionally already possible with ELF and PE executables using McSema (https://github.com/trailofbits/mcsema https://github.com/trailofbits/mcsema). If anyone is working on this please email me (address in profile), it's something we slated for a future research project and I'd love to talk.
- voltagex_ 7y ago
  >McSema and remill were developed and are maintained by Trail of Bits, funded by and used in research for DARPA and the US Department of Defense.
  - Twirrim 7y ago
    So?
    - richrichardsson 7y ago
      Don’t tell them where the internet came from, there ain’t enough tinfoil in the world.
- kenoph 7y ago
  Static binary lifting is, in general, undecidable. Actually, correct disassembly is itself undecidable, and binary lifting builds on it. Doing it dynamically, like QEMU/tcg does, is doable. It's really not too difficult to write a lifter, but it's time consuming. You have to take every instruction from the source instruction set and model its side effects in some IR. To make things simple, "whole binary" lifting tools seem to use a global CPU state (registers, flags, etc...) and so the IR you get is quite different from what you would obtain from a compiled program.
ken 7y ago
The intro makes it sound like this could be feasible for translating x86 Mac apps into ARM Mac apps. > So, what if the Mac switched to using ARM chips instead of Intel? Well, as you can see, Apple could use Bitcode to translate every Bitcode-enabled app on the Mac App Store, without consulting developers, so it would be ready to go on day one. But: > Apple doesn't enable Bitcode for submissions to the Mac App Store today Right. The flag to enable bitcode isn't even visible in Xcode for Mac apps. There's no such thing as Mac/x86 bitcode yet, in or out of the MAS. And: > ARC appears to use some inline assembly, which means you'll need to disable ARC for a project for arm64-to-x86 translation to succeed right now. So this approach will only work on Objective-C programs, written in a pre-2011 style -- which I'd bet is a vanishingly small fraction of the apps on the MAS today. It's a neat exercise in what is possible for a simplest-possible-app, but it's hard to imagine Apple would actually use this approach. It'd require them to ship a couple big new features in Xcode, and then for everyone on the MAS to recompile their apps using those features. Easier all around to just add a new architecture in Xcode the normal way, and have developers compile fat binaries.
- Someone 7y ago
  ”ARC appears to use some inline assembly, which means you'll need to disable ARC for a project for arm64-to-x86 translation to succeed right now.” That’s fairly easily fixed, certainly if you’re Apple. They already have the equivalent x86 code (Swift and modern Objective-C couldn't exist without it), so they could detect the specific inline ARM code fragment and replace it by the equivalent inline x86 code.
  - saagarjha 7y ago
    This seems like it would be very fragile.
- JonathonW 7y ago
  That's exactly my thinking-- it might be feasible for Apple to use bitcode during an x86 to ARM transition, but what problem is that actually solving? At best, Apple announces mandatory bitcode* for macOS App Store apps at WWDC in 2019; Apple announces the first ARM Macs in 2020 or 2021. So you've got a window of maybe a year or two of apps that don't have to be recompiled to support ARM, and most are likely still being updated since it hasn't been that long since they recompiled to supply Apple with a bitcode version. This doesn't do anything to solve the big headache of an architecture transition, which are those legacy apps that will never be updated again-- you'd still need an emulation solution like Rosetta or Microsoft's x86 on ARM stuff to support those applications. At best, you're maybe working around the Adobes and Microsofts of the world that move too slowly to have a day-1 update when the new architecture comes out? * I'll note that Bitcode wasn't mandatory on iOS when it was announced in 2015, and it's still not mandatory there today-- it seems unlikely that Mac Bitcode would be mandatory immediately when they announce it or in the foreseeable future.
writepub 7y ago
Alternatives: 1. WebAssembly 2. A subset of LLVM-IR that's architecture independent. PNacl from Google used this across chip architectures
- saagarjha 7y ago
  Neither of these would help migrate current Mac apps.
  - writepub 7y ago
    I meant, they are alternatives for Apple to consider, instead of reinventing the wheel on a cross architecture IR. Of course they won't help current Mac apps that don't have IR in those formats
    - saagarjha 7y ago
      I don’t think we really have a “cross architecture IR” yet.
      - writepub 7y ago
        Who's "we"? If you mean Apple, their IR (bitcode) is alomst a 100% the same as LLVM-IR, so it is not really Apple's. Aside from WebAssembly and PNacl cited in my parent comment, there's .net IR, JVM byte code, yada yada yada .. The universe is teeming with cross architecture IRs. Eventually, LLVM will have a cross platform LLVM-IR, which will possibly be even more impactful than WebAssembly has been in it's short lifespan
fiddlerwoaroof 7y ago
Sounds like we're reinventing the AS/400 piecemeal: https://en.wikipedia.org/wiki/IBM_System_i#Instruction_set https://en.wikipedia.org/wiki/IBM_System_i#Instruction_set http://ibmsystemsmag.com/blogs/you-and-i/archive/timi-protecting-investments-and-integrity-in-ibm-i/ http://ibmsystemsmag.com/blogs/you-and-i/archive/timi-protec...
saagarjha 7y ago
I would be very surprised (and impressed!) if Apple managed to pull this off. With the bitcode transition on watchOS Apple had full control over the ABI and architecture that they were going to target: this isn't the case for x86. Sure, this works for the very simple case presented here, but I forsee it quickly breaking down for any non-trivial application. At the very least, it will not work correctly on every application.
Jennie478 7y ago
Cute dog doing exercise like humans http://bit.ly/2Jq3rz1 http://bit.ly/2Jq3rz1 Funny gorilla acting like humans see his video on link below http://bit.ly/2JmTNxr http://bit.ly/2JmTNxr Best way to get rid from dark spots on face just follow this https://zoomtips.blogspot.com/2019/05/Dark-spots.html https://zoomtips.blogspot.com/2019/05/Dark-spots.html Funny panda is playing with friends they are sho cute http://bit.ly/2JvDQF4 http://bit.ly/2JvDQF4

148 points by sjuut 7y ago

This is rather "embed intermediate representation, extract, run another target's back end, and link". I was hoping this would be explaining how to go from ARM by uplifting executable to LLVM IR and retargeting for Intel. This is notionally already possible with ELF and PE executables using McSema (https://github.com/trailofbits/mcsema https://github.com/trailofbits/mcsema). If anyone is working on this please email me (address in profile), it's something we slated for a future research project and I'd love to talk.

voltagex_ 7y ago
>McSema and remill were developed and are maintained by Trail of Bits, funded by and used in research for DARPA and the US Department of Defense.
- Twirrim 7y ago
  So?
  - richrichardsson 7y ago
    Don’t tell them where the internet came from, there ain’t enough tinfoil in the world.
kenoph 7y ago
Static binary lifting is, in general, undecidable. Actually, correct disassembly is itself undecidable, and binary lifting builds on it. Doing it dynamically, like QEMU/tcg does, is doable. It's really not too difficult to write a lifter, but it's time consuming. You have to take every instruction from the source instruction set and model its side effects in some IR. To make things simple, "whole binary" lifting tools seem to use a global CPU state (registers, flags, etc...) and so the IR you get is quite different from what you would obtain from a compiled program.

ken 7y ago

The intro makes it sound like this could be feasible for translating x86 Mac apps into ARM Mac apps. > So, what if the Mac switched to using ARM chips instead of Intel? Well, as you can see, Apple could use Bitcode to translate every Bitcode-enabled app on the Mac App Store, without consulting developers, so it would be ready to go on day one. But: > Apple doesn't enable Bitcode for submissions to the Mac App Store today Right. The flag to enable bitcode isn't even visible in Xcode for Mac apps. There's no such thing as Mac/x86 bitcode yet, in or out of the MAS. And: > ARC appears to use some inline assembly, which means you'll need to disable ARC for a project for arm64-to-x86 translation to succeed right now. So this approach will only work on Objective-C programs, written in a pre-2011 style -- which I'd bet is a vanishingly small fraction of the apps on the MAS today. It's a neat exercise in what is possible for a simplest-possible-app, but it's hard to imagine Apple would actually use this approach. It'd require them to ship a couple big new features in Xcode, and then for everyone on the MAS to recompile their apps using those features. Easier all around to just add a new architecture in Xcode the normal way, and have developers compile fat binaries.

Someone 7y ago
”ARC appears to use some inline assembly, which means you'll need to disable ARC for a project for arm64-to-x86 translation to succeed right now.” That’s fairly easily fixed, certainly if you’re Apple. They already have the equivalent x86 code (Swift and modern Objective-C couldn't exist without it), so they could detect the specific inline ARM code fragment and replace it by the equivalent inline x86 code.
- saagarjha 7y ago
  This seems like it would be very fragile.
JonathonW 7y ago
That's exactly my thinking-- it might be feasible for Apple to use bitcode during an x86 to ARM transition, but what problem is that actually solving? At best, Apple announces mandatory bitcode* for macOS App Store apps at WWDC in 2019; Apple announces the first ARM Macs in 2020 or 2021. So you've got a window of maybe a year or two of apps that don't have to be recompiled to support ARM, and most are likely still being updated since it hasn't been that long since they recompiled to supply Apple with a bitcode version. This doesn't do anything to solve the big headache of an architecture transition, which are those legacy apps that will never be updated again-- you'd still need an emulation solution like Rosetta or Microsoft's x86 on ARM stuff to support those applications. At best, you're maybe working around the Adobes and Microsofts of the world that move too slowly to have a day-1 update when the new architecture comes out? * I'll note that Bitcode wasn't mandatory on iOS when it was announced in 2015, and it's still not mandatory there today-- it seems unlikely that Mac Bitcode would be mandatory immediately when they announce it or in the foreseeable future.

writepub 7y ago

Alternatives: 1. WebAssembly 2. A subset of LLVM-IR that's architecture independent. PNacl from Google used this across chip architectures

saagarjha 7y ago
Neither of these would help migrate current Mac apps.
- writepub 7y ago
  I meant, they are alternatives for Apple to consider, instead of reinventing the wheel on a cross architecture IR. Of course they won't help current Mac apps that don't have IR in those formats
  - saagarjha 7y ago
    I don’t think we really have a “cross architecture IR” yet.
    - writepub 7y ago
      Who's "we"? If you mean Apple, their IR (bitcode) is alomst a 100% the same as LLVM-IR, so it is not really Apple's. Aside from WebAssembly and PNacl cited in my parent comment, there's .net IR, JVM byte code, yada yada yada .. The universe is teeming with cross architecture IRs. Eventually, LLVM will have a cross platform LLVM-IR, which will possibly be even more impactful than WebAssembly has been in it's short lifespan

fiddlerwoaroof 7y ago

Sounds like we're reinventing the AS/400 piecemeal: https://en.wikipedia.org/wiki/IBM_System_i#Instruction_set https://en.wikipedia.org/wiki/IBM_System_i#Instruction_set http://ibmsystemsmag.com/blogs/you-and-i/archive/timi-protecting-investments-and-integrity-in-ibm-i/ http://ibmsystemsmag.com/blogs/you-and-i/archive/timi-protec...

saagarjha 7y ago

I would be very surprised (and impressed!) if Apple managed to pull this off. With the bitcode transition on watchOS Apple had full control over the ABI and architecture that they were going to target: this isn't the case for x86. Sure, this works for the very simple case presented here, but I forsee it quickly breaking down for any non-trivial application. At the very least, it will not work correctly on every application.

Jennie478 7y ago

Cute dog doing exercise like humans http://bit.ly/2Jq3rz1 http://bit.ly/2Jq3rz1 Funny gorilla acting like humans see his video on link below http://bit.ly/2JmTNxr http://bit.ly/2JmTNxr Best way to get rid from dark spots on face just follow this https://zoomtips.blogspot.com/2019/05/Dark-spots.html https://zoomtips.blogspot.com/2019/05/Dark-spots.html Funny panda is playing with friends they are sho cute http://bit.ly/2JvDQF4 http://bit.ly/2JvDQF4