5 ms·

Trying to reimplement Android without Binder is a doomed plan. Everything about Android is so intrinsically linked to having Binder available that you're just g

Trying to reimplement Android without Binder is a doomed plan. Everything about Android is so intrinsically linked to having Binder available that you're just going to end up piling hacks on hacks to have anything working. At this point, an x86 Android image will be better and more reliable. Somewhat just as fast. Services ? Binder. Intents (even internal ones) ? Binder. Play Services / microG which 90% of apps use ? Binder. Permissions ? Binder.

yjftsjthsd-h 2y ago
> At this point, an x86 Android image will be better and more reliable. Somewhat just as fast. You mean a full VM?
- sigh_again 2y ago
  No, merely using the standard Android emulator, as well as the x86_64 images that have been available for a while (or the ARM64 ones if you're on an M1 and others). With hardware acceleration properly setup, it's a nice experience. https://developer.android.com/studio/run/emulator-commandline https://developer.android.com/studio/run/emulator-commandlin... It does tend to be very unhappy when the system gets under load though, and might be sluggish if you're running a dozen IDEs on the side. (Which, as comment under says, is technically a VM. But QEMU gets a pass at pretending not to be a VM and rather just black wizardry that makes bytes happy)
  - NoahKAndrews 2y ago
    The Android Emulator is a full VM under the hood. Waydroid is a lighter-weight alternative that uses the Binder implementation in the Linux kernel.
mappu 2y ago
For most of Wine's life it was sufficient to implement the system library ABI, but recent developments mean they also have to trap the (unstable) syscall ABI with ptrace. You could mimic Binder in userspace the same way, without the kernel module. But even that's not a problem since Binder is upstream and enabled by default in many distro kernels now.
- ronsor 2y ago
  When did WINE start doing this? I always love a good read on ptrace hacks.
  - lights0123 2y ago
    https://github.com/wine-staging/wine-staging/commit/595f2f9860adafed612737529745ad24c50acdb9 https://github.com/wine-staging/wine-staging/commit/595f2f98... A new strategy is used that is much faster: https://www.phoronix.com/news/Syscall-User-Dispatch-Queued https://www.phoronix.com/news/Syscall-User-Dispatch-Queued
    - quotemstr 2y ago
      > Games avoiding the Windows API and performing system calls directly is an increasingly common occurrence by modern Windows games, seemingly in the name of Digital Rights Management schemes and similar protected modes. Yuck. Windows system call numbers are not contractual. At all. Microsoft should try harder to kill any code outside ntdll.dll that makes a system call. It's obnoxious for a developer to deliberately use unstable interfaces and then demand indefinite compatibility.
      - lxgr 2y ago
        > demand indefinite compatibility. Not infinite – long enough for the game to bring in most revenue is enough. Who cares that it stops running one or two Windows updates later for totally avoidable reasons? Definitely not the publisher. (In fact, isn't it about time for an HD remaster again?)
        quotemstr 2y ago
        Microsoft cares, though, because if a game stops working when you upgrade Windows, you blame the Windows upgrade, not Microsoft. So now Microsoft has to maintain these legacy system calls even though they never signed up to do that.
phh 2y ago
I disagree. Apps barely use Binder directly, but mostly call framework jar (which yes will do binder, but since they are reimplementing framework jar that's fine). Intents and permissions can be implemented without binder as the API is quite semantic. It may happen that the app uses Binder for same-app IPC, but since app does ser-des itself, you can easily replace Android.os.binder by whatever IPC you want, The only "real" issue I can see , and you're right there, is Play Services. I think that handling that case specifically later (once floss apps/apps that don't require play services work properly), is not too horrific (it basically needs to compile the aidl into not calling binder, but redirecting it to a custom lib)
- sigh_again 2y ago
  So, as I said, hacks on top of hacks on top of hacks :D If you're going to shim and reimplement, piece by piece some APIs like Binder... Just use Binder, that is already upstreamed in the kernel for so long.
  - phh 2y ago
    You're confusing shiming with rewriting. DirectX uses direct hardware access. DirectX uses direct hardware access. That doesn't mean you can't rewrite a DirectX-API implementation without direct hardware access. Likewise, Android's original implementation of MediaCodec uses binder. That doesn't mean that a reimplementation of MediaCodec API requires binder. The vast majority of Android public APIs do not assume binder. The only exception is Service binding, and even then, it's an assumption in name: It assumes that android.os.Binder will provide you an IPC. Most usages of android.os.Binder will be local to an app, so you can easily replace it with any IPC. Even the usage towards multiple app (whose only real usage would be Play Services) doesn't actually require binder. The requirement on that part is just the ""stable"" ser-des. As an app developer, you (should) never connect directly to /dev/binder, because its API/ABI isn't stable. (it changed like 3 times in the last 6 years?)
    - kllrnohj 2y ago
      > Most usages of android.os.Binder will be local to an app, so you can easily replace it with any IPC. I'd be absolutely shocked if that was even a little bit true. Android's binder & AIDL generated code will absolutely do in-process passthrough to avoid the syscall for local calls, but ~nobody is bothering with AIDL unless it's to enable IPC. So that local pass-through is almost certainly rarely if ever used except on "accident" when components nominally support being passed over IPC, but in some cases aren't. An example would be Android's Surface + SurfaceTexture. Normally you're using Surface over IPC either to the display or to media codec or from camera. But if you're using SurfaceTexture, then in that rare scenario it might be local to the process. But that's definitely a lot less common than IPC binder + AIDL. All that said, you could almost certainly make a Binder driver shim that just uses unix domain sockets. Heck, Android itself even has one: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/libs/binder/ndk/binder_rpc.cpp https://cs.android.com/android/platform/superproject/main/+/... What'd be a lot harder is handling some of the more advanced graphics & multimedia features. Android's gralloc HAL in particular is likely borderline impossible to implement on Linux as it assumes tight integration between all of the camera, media/video, GPU, and display stacks, and also largely assumes it's on unified memory.
refulgentis 2y ago
Binder is upstreamed into Linux. But I feel I don't know much about binder/kernel/IPC: maybe I overestimate what that means?
- yjftsjthsd-h 2y ago
  Binder is in mainline Linux, but I was under the impression that most distros aren't building it by default?
quotemstr 2y ago
Plus, Linux itself should adopt binder. It's already upstream and actually pretty good.
deaddodo 2y ago
> At this point, an x86 Android image will be better and more reliable. Somewhat just as fast. I'm pretty sure the overhead for something like Windows Subsystem Android is a few percentage points max, assuming the video card supports the proper OpenGL subsystem (so doesn't require software fallbacks). In fact, given that most x86 laptop CPUs and GPUs offer much more performance than most phone ARM chips, it's basically a moot point. The big problem is finding packages built universally or specifically for x86 (not sure what the status is for WSA on Windows ARM, but that could be an option too). I'm still waiting for the day Linux offers something similar. Or FreeBSD does something similar to their Linux ABI compatibility. Edit: looks like WSA is being ended in early 2025: https://learn.microsoft.com/en-us/windows/android/wsa/ https://learn.microsoft.com/en-us/windows/android/wsa/ Due to not being able to integrate a store and make revenuue: https://www.windowscentral.com/software-apps/windows-11/heres-why-windows-subsystem-for-android-got-killed-off-according-to-a-29-year-microsoft-veteran https://www.windowscentral.com/software-apps/windows-11/here... What a shame.
- yjftsjthsd-h 2y ago
  > I'm still waiting for the day Linux offers something similar. Waiting? Waydroid is several years old at this point, and you've always been able to run Android in a VM via either Google's official device emulator in the SDK or just running https://www.android-x86.org/ https://www.android-x86.org/ et al. in whatever hypervisor you prefer.
  - cyberax 2y ago
    Android-x86 is dead. The last commits are from years ago. Waydroid is slightly more alive, but only barely. And it itself is a fork of now defunct Anbox.
  - deaddodo 2y ago
    Sorry, I should have rephrased that: I'm still waiting for Linux to offer a good version of that. I currently use waydroid, and it's barely functional.
- surajrmal 2y ago
pjmlp 2y ago
Drivers? Binder. App extensions? Binder.
stuaxo 2y ago
I always have trouble with AnBox and Waydroid - when it comes to Binder. It seems like there were changes made to Binder between the version in Linux mainline and the one in Android. Until that is properly synched up then using something else will be easier.

yjftsjthsd-h 2y ago

> At this point, an x86 Android image will be better and more reliable. Somewhat just as fast. You mean a full VM?

sigh_again 2y ago
No, merely using the standard Android emulator, as well as the x86_64 images that have been available for a while (or the ARM64 ones if you're on an M1 and others). With hardware acceleration properly setup, it's a nice experience. https://developer.android.com/studio/run/emulator-commandline https://developer.android.com/studio/run/emulator-commandlin... It does tend to be very unhappy when the system gets under load though, and might be sluggish if you're running a dozen IDEs on the side. (Which, as comment under says, is technically a VM. But QEMU gets a pass at pretending not to be a VM and rather just black wizardry that makes bytes happy)
- NoahKAndrews 2y ago
  The Android Emulator is a full VM under the hood. Waydroid is a lighter-weight alternative that uses the Binder implementation in the Linux kernel.

mappu 2y ago

For most of Wine's life it was sufficient to implement the system library ABI, but recent developments mean they also have to trap the (unstable) syscall ABI with ptrace. You could mimic Binder in userspace the same way, without the kernel module. But even that's not a problem since Binder is upstream and enabled by default in many distro kernels now.

ronsor 2y ago
When did WINE start doing this? I always love a good read on ptrace hacks.
- lights0123 2y ago
  https://github.com/wine-staging/wine-staging/commit/595f2f9860adafed612737529745ad24c50acdb9 https://github.com/wine-staging/wine-staging/commit/595f2f98... A new strategy is used that is much faster: https://www.phoronix.com/news/Syscall-User-Dispatch-Queued https://www.phoronix.com/news/Syscall-User-Dispatch-Queued
  - quotemstr 2y ago
    > Games avoiding the Windows API and performing system calls directly is an increasingly common occurrence by modern Windows games, seemingly in the name of Digital Rights Management schemes and similar protected modes. Yuck. Windows system call numbers are not contractual. At all. Microsoft should try harder to kill any code outside ntdll.dll that makes a system call. It's obnoxious for a developer to deliberately use unstable interfaces and then demand indefinite compatibility.
    - lxgr 2y ago
      > demand indefinite compatibility. Not infinite – long enough for the game to bring in most revenue is enough. Who cares that it stops running one or two Windows updates later for totally avoidable reasons? Definitely not the publisher. (In fact, isn't it about time for an HD remaster again?)
      - quotemstr 2y ago
        Microsoft cares, though, because if a game stops working when you upgrade Windows, you blame the Windows upgrade, not Microsoft. So now Microsoft has to maintain these legacy system calls even though they never signed up to do that.

phh 2y ago

I disagree. Apps barely use Binder directly, but mostly call framework jar (which yes will do binder, but since they are reimplementing framework jar that's fine). Intents and permissions can be implemented without binder as the API is quite semantic. It may happen that the app uses Binder for same-app IPC, but since app does ser-des itself, you can easily replace Android.os.binder by whatever IPC you want, The only "real" issue I can see , and you're right there, is Play Services. I think that handling that case specifically later (once floss apps/apps that don't require play services work properly), is not too horrific (it basically needs to compile the aidl into not calling binder, but redirecting it to a custom lib)

sigh_again 2y ago
So, as I said, hacks on top of hacks on top of hacks :D If you're going to shim and reimplement, piece by piece some APIs like Binder... Just use Binder, that is already upstreamed in the kernel for so long.
- phh 2y ago
  You're confusing shiming with rewriting. DirectX uses direct hardware access. DirectX uses direct hardware access. That doesn't mean you can't rewrite a DirectX-API implementation without direct hardware access. Likewise, Android's original implementation of MediaCodec uses binder. That doesn't mean that a reimplementation of MediaCodec API requires binder. The vast majority of Android public APIs do not assume binder. The only exception is Service binding, and even then, it's an assumption in name: It assumes that android.os.Binder will provide you an IPC. Most usages of android.os.Binder will be local to an app, so you can easily replace it with any IPC. Even the usage towards multiple app (whose only real usage would be Play Services) doesn't actually require binder. The requirement on that part is just the ""stable"" ser-des. As an app developer, you (should) never connect directly to /dev/binder, because its API/ABI isn't stable. (it changed like 3 times in the last 6 years?)
  - kllrnohj 2y ago
    > Most usages of android.os.Binder will be local to an app, so you can easily replace it with any IPC. I'd be absolutely shocked if that was even a little bit true. Android's binder & AIDL generated code will absolutely do in-process passthrough to avoid the syscall for local calls, but ~nobody is bothering with AIDL unless it's to enable IPC. So that local pass-through is almost certainly rarely if ever used except on "accident" when components nominally support being passed over IPC, but in some cases aren't. An example would be Android's Surface + SurfaceTexture. Normally you're using Surface over IPC either to the display or to media codec or from camera. But if you're using SurfaceTexture, then in that rare scenario it might be local to the process. But that's definitely a lot less common than IPC binder + AIDL. All that said, you could almost certainly make a Binder driver shim that just uses unix domain sockets. Heck, Android itself even has one: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/libs/binder/ndk/binder_rpc.cpp https://cs.android.com/android/platform/superproject/main/+/... What'd be a lot harder is handling some of the more advanced graphics & multimedia features. Android's gralloc HAL in particular is likely borderline impossible to implement on Linux as it assumes tight integration between all of the camera, media/video, GPU, and display stacks, and also largely assumes it's on unified memory.

refulgentis 2y ago

Binder is upstreamed into Linux. But I feel I don't know much about binder/kernel/IPC: maybe I overestimate what that means?

yjftsjthsd-h 2y ago
Binder is in mainline Linux, but I was under the impression that most distros aren't building it by default?

quotemstr 2y ago

Plus, Linux itself should adopt binder. It's already upstream and actually pretty good.

deaddodo 2y ago

> At this point, an x86 Android image will be better and more reliable. Somewhat just as fast. I'm pretty sure the overhead for something like Windows Subsystem Android is a few percentage points max, assuming the video card supports the proper OpenGL subsystem (so doesn't require software fallbacks). In fact, given that most x86 laptop CPUs and GPUs offer much more performance than most phone ARM chips, it's basically a moot point. The big problem is finding packages built universally or specifically for x86 (not sure what the status is for WSA on Windows ARM, but that could be an option too). I'm still waiting for the day Linux offers something similar. Or FreeBSD does something similar to their Linux ABI compatibility. Edit: looks like WSA is being ended in early 2025: https://learn.microsoft.com/en-us/windows/android/wsa/ https://learn.microsoft.com/en-us/windows/android/wsa/ Due to not being able to integrate a store and make revenuue: https://www.windowscentral.com/software-apps/windows-11/heres-why-windows-subsystem-for-android-got-killed-off-according-to-a-29-year-microsoft-veteran https://www.windowscentral.com/software-apps/windows-11/here... What a shame.

yjftsjthsd-h 2y ago
> I'm still waiting for the day Linux offers something similar. Waiting? Waydroid is several years old at this point, and you've always been able to run Android in a VM via either Google's official device emulator in the SDK or just running https://www.android-x86.org/ https://www.android-x86.org/ et al. in whatever hypervisor you prefer.
- cyberax 2y ago
  Android-x86 is dead. The last commits are from years ago. Waydroid is slightly more alive, but only barely. And it itself is a fork of now defunct Anbox.
- deaddodo 2y ago
  Sorry, I should have rephrased that: I'm still waiting for Linux to offer a good version of that. I currently use waydroid, and it's barely functional.
surajrmal 2y ago

pjmlp 2y ago

Drivers? Binder. App extensions? Binder.

stuaxo 2y ago

I always have trouble with AnBox and Waydroid - when it comes to Binder. It seems like there were changes made to Binder between the version in Linux mainline and the one in Android. Until that is properly synched up then using something else will be easier.