6 ms·

It is very likely that ARM-based Macs will lack a performant hypervisor upon release. We will have to see how VMWare responds. I'd bet it will inspire new produ

It is very likely that ARM-based Macs will lack a performant hypervisor upon release. We will have to see how VMWare responds. I'd bet it will inspire new products and innovation and the desktop space will move towards a less x86-x64 centric world. In the end it is a short term problem. Someone will respond and provide a performant hypervisor that can run on an ARM host and virtualize x86-x64 and ARM guests. It's true it will cause some pain in the first year or two, but even as a heavy VMWare Fusion user I am really looking forward to the benefits of a vertically integrated laptop.

WrtCdEvrydy 6y ago
VMWare did have a hypervisor for Raspberry Pi.
- my123 6y ago
  Yes, VMWare ESXi runs on Arm just fine. :-)
  - jki275 6y ago
    Can you run an x86 guest under it?
    - lode 6y ago
      No. Virtualization (dividing a host into different logical hosts but executing unmodified CPU instructions, like VMware, VirtualBox, ...) and emulation (translating instructions, like Rosetta) are two different beasts.
      - jki275 6y ago
        That's pretty much what I assumed from what I know of VMWare. It's going to be a big issue for future Macs, there are entire segments of developers who may have to abandon Macs if we can't run VMs of x86 operating systems.
sirn 6y ago
Apple has Hypervisor.framework which has been updated for ARM Mac[1]. Xhyve and HyperKit (used by Docker for Mac) uses Hypervisor.framework exclusively. The last time I tried Hypervisor.framework on x86-64, the CPU performance was quite fine (matches that of VMware/VirtualBox), but I/O was pretty abysmal. Emulating x86-64 on ARM is probably going to be a role of something similar to QEMU. [1]: https://developer.apple.com/documentation/hypervisor/apple_silicon https://developer.apple.com/documentation/hypervisor/apple_s...
- tw04 6y ago
  Parallels already announced they’re going to have an option. TBD on what it looks like. https://www.parallels.com/blogs/apple-silicon-wwdc/ https://www.parallels.com/blogs/apple-silicon-wwdc/
  - thedevelopnik 6y ago
    They showed it running Debian in the keynote, but it was ARM Debian. It’s gonna be awhile, if ever, til we get x86 support on Apple Silicon.
- john_alan 6y ago
  Cool! Does docker no longer use virtual box on macOS?
  - sirn 6y ago
    Depends on whether you install Docker for Mac (uses HyperKit) or Docker Toolbox (uses VirtualBox)
  - bmalehorn 6y ago
    Based on this article, it now uses Hypervisor.framework: https://www.quora.com/What-hypervisor-does-Docker-use https://www.quora.com/What-hypervisor-does-Docker-use > In Linux, Docker uses kernel features. It does not use any hypervisor. > In Windows, Docker desktop uses Hyper-V. > In Mac, Docker desktop uses Apple Hypervisor.framework.
ajconway 6y ago
If I understand correctly, hypervisors don't emulate hardware, that's what emulators (like QEMU) do. That would mean that physically the most performant option to run x86 code on an ARM CPU is dynamic translation (like QEMU-TCG or the new Rosetta JIT support).
- saagarjha 6y ago
  Rosetta 2 is ideally a static binary transformation, it only falls back to emulation when this doesn't work. So it's a bit different than TCG :)
- timsally 6y ago
  Broadly speaking yes. Generally, hypervisors mediate access to shared hardware whereas emulators implement simulated hardware in software. The very first hypervisors worked using dynamic binary translation. They would run a "guest" operating system by executing a stream of native instructions directly on the host CPU. This stream would be dynamically translated to remove and trap in software any privileged operations so the hypervisor could handle them. Modern hypervisors take advantage of hardware features that allow you to more efficiently trap on privileged operations. ARM started to add some of these features starting in 2013 [1]. In contrast Intel first started adding these features to the Pentium 4 in 2005 [2]. When such hardware features were released, they actually were not faster than the software translation. These days the hardware based options are faster. There is even hardware support for running nested hypervisors. So the first question we need to ask is how hypervisors implemented with ARMs hardware features stack up to Intel. I have no doubt that parity at a minimum will be reached I just don't know what the current state of play is. As indicated in my original comment, if I had to bet on release we wont quite have the performance or feature set you would be used to with a product like VMWare Fusion. The second question we need to ask is whether there is a way to efficiently emulate x86-64 processors on ARM hosts. Even better if you can do this while taking advantage of the supporting infrastructure hypervisors already have in terms of the emulated devices and other features. QEMU just gets you the CPU and a short list of a devices. The fully experience of a seamlessly virtualized guest requires a lot more than that. But at the core you are right that it is going to require QEMU-TCG, Rosetta 2, or some similar technology because the silicon just is not there to execute x86-64. Exciting stuff! We'll see where it all lands. [1] https://lwn.net/Articles/557132/ https://lwn.net/Articles/557132/ [2] https://en.wikipedia.org/wiki/X86_virtualization#Intel-VT-x https://en.wikipedia.org/wiki/X86_virtualization#Intel-VT-x
mister_hn 6y ago
With just less than 10% of market share, do you really think it will change the whole thing? Unless Microsoft pushes for ARM too, I don't see any changes soon
- timsally 6y ago
  We'll have to see. As another person already pointed out, VMWare has experimented with ESXi on ARM and they claim their customers could realize significant cost savings by migrating to ARM [1]. So if they've already done a good amount of engineering work on it, we may well see VMWare Fusion on ARM that can efficiently run ARM guests. They plan on releasing a tech preview in July [2]. Whether you can stick an emulated x86-64 CPU in there is another matter. It's a much bigger engineering lift and unless Apple puts some resources into it it's not clear to me a virtualization company by themselves would want to incur the cost. I hope there is enough demand for it and that someone will provide it. For me personally the only reason I run VMWare Fusion is to access x86-only Windows applications for which there is no replacement. [1] https://blogs.vmware.com/vsphere/2019/10/esxi-on-arm-at-the-edge-on-the-smartnic-and-in-the-cloud.html https://blogs.vmware.com/vsphere/2019/10/esxi-on-arm-at-the-.... [2] https://twitter.com/VMwareFusion/status/1275466832002945024 https://twitter.com/VMwareFusion/status/1275466832002945024
- cwhiz 6y ago
  Microsoft has been dabbling with ARM for a long time now. It will all come down to whether this move gives Apple a significant performance and/or battery life advantage. If Apple pulls it off it will force Microsoft and other vendors to respond.
  - mister_hn 6y ago
    They announced there will be no Bootcamp, I hope there will be a possibility to install other OSes as well on their new machines. This is critical especially when they make their older devices "EOL" and there are no more OS updates.
    - cwhiz 6y ago

WrtCdEvrydy 6y ago

VMWare did have a hypervisor for Raspberry Pi.

my123 6y ago
Yes, VMWare ESXi runs on Arm just fine. :-)
- jki275 6y ago
  Can you run an x86 guest under it?
  - lode 6y ago
    No. Virtualization (dividing a host into different logical hosts but executing unmodified CPU instructions, like VMware, VirtualBox, ...) and emulation (translating instructions, like Rosetta) are two different beasts.
    - jki275 6y ago
      That's pretty much what I assumed from what I know of VMWare. It's going to be a big issue for future Macs, there are entire segments of developers who may have to abandon Macs if we can't run VMs of x86 operating systems.

sirn 6y ago

Apple has Hypervisor.framework which has been updated for ARM Mac[1]. Xhyve and HyperKit (used by Docker for Mac) uses Hypervisor.framework exclusively. The last time I tried Hypervisor.framework on x86-64, the CPU performance was quite fine (matches that of VMware/VirtualBox), but I/O was pretty abysmal. Emulating x86-64 on ARM is probably going to be a role of something similar to QEMU. [1]: https://developer.apple.com/documentation/hypervisor/apple_silicon https://developer.apple.com/documentation/hypervisor/apple_s...

tw04 6y ago
Parallels already announced they’re going to have an option. TBD on what it looks like. https://www.parallels.com/blogs/apple-silicon-wwdc/ https://www.parallels.com/blogs/apple-silicon-wwdc/
- thedevelopnik 6y ago
  They showed it running Debian in the keynote, but it was ARM Debian. It’s gonna be awhile, if ever, til we get x86 support on Apple Silicon.
john_alan 6y ago
Cool! Does docker no longer use virtual box on macOS?
- sirn 6y ago
  Depends on whether you install Docker for Mac (uses HyperKit) or Docker Toolbox (uses VirtualBox)
- bmalehorn 6y ago
  Based on this article, it now uses Hypervisor.framework: https://www.quora.com/What-hypervisor-does-Docker-use https://www.quora.com/What-hypervisor-does-Docker-use > In Linux, Docker uses kernel features. It does not use any hypervisor. > In Windows, Docker desktop uses Hyper-V. > In Mac, Docker desktop uses Apple Hypervisor.framework.

ajconway 6y ago

If I understand correctly, hypervisors don't emulate hardware, that's what emulators (like QEMU) do. That would mean that physically the most performant option to run x86 code on an ARM CPU is dynamic translation (like QEMU-TCG or the new Rosetta JIT support).

saagarjha 6y ago
Rosetta 2 is ideally a static binary transformation, it only falls back to emulation when this doesn't work. So it's a bit different than TCG :)
timsally 6y ago
Broadly speaking yes. Generally, hypervisors mediate access to shared hardware whereas emulators implement simulated hardware in software. The very first hypervisors worked using dynamic binary translation. They would run a "guest" operating system by executing a stream of native instructions directly on the host CPU. This stream would be dynamically translated to remove and trap in software any privileged operations so the hypervisor could handle them. Modern hypervisors take advantage of hardware features that allow you to more efficiently trap on privileged operations. ARM started to add some of these features starting in 2013 [1]. In contrast Intel first started adding these features to the Pentium 4 in 2005 [2]. When such hardware features were released, they actually were not faster than the software translation. These days the hardware based options are faster. There is even hardware support for running nested hypervisors. So the first question we need to ask is how hypervisors implemented with ARMs hardware features stack up to Intel. I have no doubt that parity at a minimum will be reached I just don't know what the current state of play is. As indicated in my original comment, if I had to bet on release we wont quite have the performance or feature set you would be used to with a product like VMWare Fusion. The second question we need to ask is whether there is a way to efficiently emulate x86-64 processors on ARM hosts. Even better if you can do this while taking advantage of the supporting infrastructure hypervisors already have in terms of the emulated devices and other features. QEMU just gets you the CPU and a short list of a devices. The fully experience of a seamlessly virtualized guest requires a lot more than that. But at the core you are right that it is going to require QEMU-TCG, Rosetta 2, or some similar technology because the silicon just is not there to execute x86-64. Exciting stuff! We'll see where it all lands. [1] https://lwn.net/Articles/557132/ https://lwn.net/Articles/557132/ [2] https://en.wikipedia.org/wiki/X86_virtualization#Intel-VT-x https://en.wikipedia.org/wiki/X86_virtualization#Intel-VT-x

mister_hn 6y ago

With just less than 10% of market share, do you really think it will change the whole thing? Unless Microsoft pushes for ARM too, I don't see any changes soon

timsally 6y ago
We'll have to see. As another person already pointed out, VMWare has experimented with ESXi on ARM and they claim their customers could realize significant cost savings by migrating to ARM [1]. So if they've already done a good amount of engineering work on it, we may well see VMWare Fusion on ARM that can efficiently run ARM guests. They plan on releasing a tech preview in July [2]. Whether you can stick an emulated x86-64 CPU in there is another matter. It's a much bigger engineering lift and unless Apple puts some resources into it it's not clear to me a virtualization company by themselves would want to incur the cost. I hope there is enough demand for it and that someone will provide it. For me personally the only reason I run VMWare Fusion is to access x86-only Windows applications for which there is no replacement. [1] https://blogs.vmware.com/vsphere/2019/10/esxi-on-arm-at-the-edge-on-the-smartnic-and-in-the-cloud.html https://blogs.vmware.com/vsphere/2019/10/esxi-on-arm-at-the-.... [2] https://twitter.com/VMwareFusion/status/1275466832002945024 https://twitter.com/VMwareFusion/status/1275466832002945024
cwhiz 6y ago
Microsoft has been dabbling with ARM for a long time now. It will all come down to whether this move gives Apple a significant performance and/or battery life advantage. If Apple pulls it off it will force Microsoft and other vendors to respond.
- mister_hn 6y ago
  They announced there will be no Bootcamp, I hope there will be a possibility to install other OSes as well on their new machines. This is critical especially when they make their older devices "EOL" and there are no more OS updates.
  - cwhiz 6y ago