13 ms·

Most people would be better off waiting for the multiple RVA23 boards that are supposed to come out this year, at least if they don't want to be stuck running c

by alexrp 8mo ago

Most people would be better off waiting for the multiple RVA23 boards that are supposed to come out this year, at least if they don't want to be stuck running custom vendor distros. "RVA23 except V" at this price point and at this point in time is a pretty bad value proposition. It's honestly a bit hard to understand why they bothered with this one. No hate for the Milk-V folks; I have 4 Jupiters sitting next to me running in Zig's CI. But hopefully they'll have something RVA23-compliant out soon (SpacemiT K3?).

irusensei 8mo ago
I feel this is becoming a bit of a tech urban legend such as ZFS requires ECC. As far as I understand the RVA23 requirement is an ubuntu thing only and only for current non LTS and future releases. Current LTS doesn't have such requirements and neither other distributions such as Fedora and Debian that support riscv64. So no, you are not stuck running custom vendor distros because of this but more because the other weird device drivers and boot systems that have no mainline support.
- alexrp 8mo ago
  I'm fairly sure I recall Fedora folks signaling that they intend to move to RVA23 as soon as hardware becomes generally available. It is of course possible that Debian sticks with RV64GC for the long term, but I seriously doubt it. It's just too much performance to leave on the table for a relatively new port, especially when RVA23 will (very) soon be the expected baseline for general-purpose RISC-V systems.
  - rwmj 8mo ago
    As someone from the Fedora/RISC-V project, it'll depend on what our users want. We cannot support both RV64GC and RVA23 (because we don't have the build or software infra to do it) so we have to be careful when we move. Doing something like building with RV64GC generally but having targeted optimizations - perhaps two kernel variants and some libraries - might be possible, but also isn't easy. Things are different for CentOS / RHEL where we'll be able to move to RVA23 (and beyond) much more aggressively.
    - znpy 8mo ago
      First things first: thank you for your work. That being said: does it make sense to keep a nee but low performance platform alive? As the platform is new and likely doesn’t have many users, wouldn’t it make sense to nudge (as in “gently push”) users towards a higher performance platform? Chances are the low-performance platform will die anyway, and fedora will not be exploiting the full offering of the high performance platform.
      - rwmj 8mo ago
        It's about what users think in our forums: https://discussion.fedoraproject.org/tag/risc-v-sig https://discussion.fedoraproject.org/tag/risc-v-sig
- fweimer 8mo ago
  I'm not completely sure, but I suspect Fedora will stick to the current baseline for quite some time. But the baseline is quite minimal. It's biased towards efficient emulation of the instructions in portable C code. I'm not sure why anyone would target an enterprise distribution to that. On the other hand, even RVA23 is quite poor at signed overflow checking. Like MIPS before it, RISC-V is a bet that we're going to write software in C-like languages for a long time.
  - IshKebab 8mo ago
    > On the other hand, even RVA23 is quite poor at signed overflow checking. On the other hand it avoids integer flags which is nice. I doubt it makes a measurable performance impact either way on modern OoO CPUs. There's going to be no data dependence on the extra instructions needed to calculate overflow except for the branch, which will be predicted not-taken, so the other instructions after it will basically always run speculatively in parallel with the overflow-checking instructions.
    - fweimer 8mo ago
      It's nice for a C simulator to avoid condition codes. It's not so nice if you want consistent overflow checks (e.g., for automatically overflowing from fixnums to bignums). Even with XNOR (which isn't even part of RVA23, if I recall correctly), the sequence for doing an overflow check is quite messy. On AArch64 and x86-64, it's just the operation followed by a conditional jump: https://godbolt.org/z/968Eb1dh1 https://godbolt.org/z/968Eb1dh1
      - adgjlsfhk1 8mo ago
        Non-flag based overflow checks are still pretty cheap. The overflow check is only 1 extra instruction for unsigned (both add and multiply), and 3/4 extra for signed overflow (see https://godbolt.org/z/nq1nb5Whr https://godbolt.org/z/nq1nb5Whr for details). It's also worth noting that in many cases, the overflow checks will be removable or simplify-able by the compiler entirely (e.g. if you're adding 1 or know the sign of one of the operands etc). As such, the extra couple instructions are likely worthwhile if it makes designing a wider core easier. Signed overflow instructions would be reasonable to add, but it's not like modern high performance cores are bottlenecked by scalar instructions that don't touch memory anyway.
IshKebab 8mo ago
I don't think you'll be able to get away from custom distros even with RVA23. It solves the problem of binary compatibility - everything compiled for RVA23 should be pretty portable at the instruction level (won't help with the usual glibc nonsense of course). But RVA23 doesn't help with the hardware layer - it's going to be exactly the same as ARM SBCs where there's no hardware discovery mechanism and everything has to be hard-coded in the Linux device tree. You still need a custom distro for Raspberry Pi for example. I believe there has been some progress in getting RISC-V ACPI support, and there's at least the intent of making mconfigptr do something useful - for a while there was a "unified discovery" task group, but it seems like there just wasn't enough manpower behind it and it disbanded. https://github.com/riscvarchive/configuration-structure/blob/master/charter.pdf https://github.com/riscvarchive/configuration-structure/blob... https://riscv.atlassian.net/browse/RVG-50 https://riscv.atlassian.net/browse/RVG-50
- alexrp 8mo ago
  > You still need a custom distro for Raspberry Pi for example. Are you sure that's still the case? I just checked the Raspberry Pi Imager and I see several "stock" distro options that aren't Raspbian. Regardless, I take your point that we're reliant on vendors actually doing the upstreaming work for device trees (and drivers). But so far the recognizable players in the RISC-V space do all(?) seem to be doing that, so for now I remain hopeful that we can avoid the Arm mess.
  - IshKebab 8mo ago
    I'm not totally sure, but I would imagine those stock distros actually have dedicated packages for Raspberry Pi kernel images. See this for example: https://www.phoronix.com/news/Raspberry-Pi-5-Ethernet-Linux https://www.phoronix.com/news/Raspberry-Pi-5-Ethernet-Linux If you look at the patch series, it's directly adding information about the address of the ethernet device. That's the sort of thing that would be discovered automatically in the x86 world. It wouldn't need to be hard-coded into the kernel for each individual board that is supported.
camel-cdr 8mo ago
> But hopefully they'll have something RVA23-compliant out soon (SpacemiT K3?). A handful of developers already have access to SpacemiT K3 hardware, which is indeed RVA23 compliant and already runs Ubuntu 26.04. geekbench: https://browser.geekbench.com/v6/cpu/16145076 https://browser.geekbench.com/v6/cpu/16145076 rvv-bench: https://camel-cdr.github.io/rvv-bench-results/spacemit_x100/index.html https://camel-cdr.github.io/rvv-bench-results/spacemit_x100/... (which as instruction throughput measurements and more)
- ahoka 8mo ago
  This is around the performance of a Core 2 Duo, if I understand correctly?
  - camel-cdr 8mo ago
    The single core performance is roughly in the middle between Pi4 Cortex-A72 and Pi5 Cortex-A76. It's slightly faster than a 3GHz Core 2 Dua in scalar single threaded performance, but it has 8 cores instead of two and more SIMD performance. There are also 8 additional SpacemiT-A100 cores with 1024-bit wide vectors, which are more like an additional accelerator. The geekbench score is a bit lower than it should be, because at least three benchmarks are still missing SIMD acceleration on RISC-V (File Compression, Asset Compression, Ray Tracer), and the HTML5 browser test is also missing optimizations. I'd estimate it should be able to get to the 500 range with comparable optimization to other architectures. The Milk-V Titan mention in the original post is actually slightly faster in scalar performance, but has no RISC-V Vector support at all, which causes it's geekbench score to be way lower.
    - oxxoxoxooo 8mo ago
      Do you happen to know how does one access/use those A100 cores?
      - camel-cdr 8mo ago
        No. The problem is that you can't migrate threads between cores with different vector length. The current ubuntu 26.04 image, that is installed, lists 16 cores in htop, but you can only run applications on the first 8 (e.g. taskset -c 10 fails). If you query whats running on the A100 cores you see things like a "kworker" processes. I suspect that it should be possible to write a custom kernel module that runs on the A100s with the current kernel, but I'm not sure. I expect it will definitely be possible to boot a OS only one the 8 A100 cores. Well have to see if they manage to figure out how to add support for explicitly pinning user mode processes to the cores. The ideal configuration would be to have everything run only on the X100s, but with an opt-in mechanism to run a program only on an A100 core.

by alexrp 8mo ago

irusensei 8mo ago

I feel this is becoming a bit of a tech urban legend such as ZFS requires ECC. As far as I understand the RVA23 requirement is an ubuntu thing only and only for current non LTS and future releases. Current LTS doesn't have such requirements and neither other distributions such as Fedora and Debian that support riscv64. So no, you are not stuck running custom vendor distros because of this but more because the other weird device drivers and boot systems that have no mainline support.

alexrp 8mo ago
I'm fairly sure I recall Fedora folks signaling that they intend to move to RVA23 as soon as hardware becomes generally available. It is of course possible that Debian sticks with RV64GC for the long term, but I seriously doubt it. It's just too much performance to leave on the table for a relatively new port, especially when RVA23 will (very) soon be the expected baseline for general-purpose RISC-V systems.
- rwmj 8mo ago
  As someone from the Fedora/RISC-V project, it'll depend on what our users want. We cannot support both RV64GC and RVA23 (because we don't have the build or software infra to do it) so we have to be careful when we move. Doing something like building with RV64GC generally but having targeted optimizations - perhaps two kernel variants and some libraries - might be possible, but also isn't easy. Things are different for CentOS / RHEL where we'll be able to move to RVA23 (and beyond) much more aggressively.
  - znpy 8mo ago
    First things first: thank you for your work. That being said: does it make sense to keep a nee but low performance platform alive? As the platform is new and likely doesn’t have many users, wouldn’t it make sense to nudge (as in “gently push”) users towards a higher performance platform? Chances are the low-performance platform will die anyway, and fedora will not be exploiting the full offering of the high performance platform.
    - rwmj 8mo ago
      It's about what users think in our forums: https://discussion.fedoraproject.org/tag/risc-v-sig https://discussion.fedoraproject.org/tag/risc-v-sig
fweimer 8mo ago
I'm not completely sure, but I suspect Fedora will stick to the current baseline for quite some time. But the baseline is quite minimal. It's biased towards efficient emulation of the instructions in portable C code. I'm not sure why anyone would target an enterprise distribution to that. On the other hand, even RVA23 is quite poor at signed overflow checking. Like MIPS before it, RISC-V is a bet that we're going to write software in C-like languages for a long time.
- IshKebab 8mo ago
  > On the other hand, even RVA23 is quite poor at signed overflow checking. On the other hand it avoids integer flags which is nice. I doubt it makes a measurable performance impact either way on modern OoO CPUs. There's going to be no data dependence on the extra instructions needed to calculate overflow except for the branch, which will be predicted not-taken, so the other instructions after it will basically always run speculatively in parallel with the overflow-checking instructions.
  - fweimer 8mo ago
    It's nice for a C simulator to avoid condition codes. It's not so nice if you want consistent overflow checks (e.g., for automatically overflowing from fixnums to bignums). Even with XNOR (which isn't even part of RVA23, if I recall correctly), the sequence for doing an overflow check is quite messy. On AArch64 and x86-64, it's just the operation followed by a conditional jump: https://godbolt.org/z/968Eb1dh1 https://godbolt.org/z/968Eb1dh1
    - adgjlsfhk1 8mo ago
      Non-flag based overflow checks are still pretty cheap. The overflow check is only 1 extra instruction for unsigned (both add and multiply), and 3/4 extra for signed overflow (see https://godbolt.org/z/nq1nb5Whr https://godbolt.org/z/nq1nb5Whr for details). It's also worth noting that in many cases, the overflow checks will be removable or simplify-able by the compiler entirely (e.g. if you're adding 1 or know the sign of one of the operands etc). As such, the extra couple instructions are likely worthwhile if it makes designing a wider core easier. Signed overflow instructions would be reasonable to add, but it's not like modern high performance cores are bottlenecked by scalar instructions that don't touch memory anyway.

IshKebab 8mo ago

I don't think you'll be able to get away from custom distros even with RVA23. It solves the problem of binary compatibility - everything compiled for RVA23 should be pretty portable at the instruction level (won't help with the usual glibc nonsense of course). But RVA23 doesn't help with the hardware layer - it's going to be exactly the same as ARM SBCs where there's no hardware discovery mechanism and everything has to be hard-coded in the Linux device tree. You still need a custom distro for Raspberry Pi for example. I believe there has been some progress in getting RISC-V ACPI support, and there's at least the intent of making mconfigptr do something useful - for a while there was a "unified discovery" task group, but it seems like there just wasn't enough manpower behind it and it disbanded. https://github.com/riscvarchive/configuration-structure/blob/master/charter.pdf https://github.com/riscvarchive/configuration-structure/blob... https://riscv.atlassian.net/browse/RVG-50 https://riscv.atlassian.net/browse/RVG-50

alexrp 8mo ago
> You still need a custom distro for Raspberry Pi for example. Are you sure that's still the case? I just checked the Raspberry Pi Imager and I see several "stock" distro options that aren't Raspbian. Regardless, I take your point that we're reliant on vendors actually doing the upstreaming work for device trees (and drivers). But so far the recognizable players in the RISC-V space do all(?) seem to be doing that, so for now I remain hopeful that we can avoid the Arm mess.
- IshKebab 8mo ago
  I'm not totally sure, but I would imagine those stock distros actually have dedicated packages for Raspberry Pi kernel images. See this for example: https://www.phoronix.com/news/Raspberry-Pi-5-Ethernet-Linux https://www.phoronix.com/news/Raspberry-Pi-5-Ethernet-Linux If you look at the patch series, it's directly adding information about the address of the ethernet device. That's the sort of thing that would be discovered automatically in the x86 world. It wouldn't need to be hard-coded into the kernel for each individual board that is supported.

camel-cdr 8mo ago

> But hopefully they'll have something RVA23-compliant out soon (SpacemiT K3?). A handful of developers already have access to SpacemiT K3 hardware, which is indeed RVA23 compliant and already runs Ubuntu 26.04. geekbench: https://browser.geekbench.com/v6/cpu/16145076 https://browser.geekbench.com/v6/cpu/16145076 rvv-bench: https://camel-cdr.github.io/rvv-bench-results/spacemit_x100/index.html https://camel-cdr.github.io/rvv-bench-results/spacemit_x100/... (which as instruction throughput measurements and more)

ahoka 8mo ago
This is around the performance of a Core 2 Duo, if I understand correctly?
- camel-cdr 8mo ago
  The single core performance is roughly in the middle between Pi4 Cortex-A72 and Pi5 Cortex-A76. It's slightly faster than a 3GHz Core 2 Dua in scalar single threaded performance, but it has 8 cores instead of two and more SIMD performance. There are also 8 additional SpacemiT-A100 cores with 1024-bit wide vectors, which are more like an additional accelerator. The geekbench score is a bit lower than it should be, because at least three benchmarks are still missing SIMD acceleration on RISC-V (File Compression, Asset Compression, Ray Tracer), and the HTML5 browser test is also missing optimizations. I'd estimate it should be able to get to the 500 range with comparable optimization to other architectures. The Milk-V Titan mention in the original post is actually slightly faster in scalar performance, but has no RISC-V Vector support at all, which causes it's geekbench score to be way lower.
  - oxxoxoxooo 8mo ago
    Do you happen to know how does one access/use those A100 cores?
    - camel-cdr 8mo ago
      No. The problem is that you can't migrate threads between cores with different vector length. The current ubuntu 26.04 image, that is installed, lists 16 cores in htop, but you can only run applications on the first 8 (e.g. taskset -c 10 fails). If you query whats running on the A100 cores you see things like a "kworker" processes. I suspect that it should be possible to write a custom kernel module that runs on the A100s with the current kernel, but I'm not sure. I expect it will definitely be possible to boot a OS only one the 8 A100 cores. Well have to see if they manage to figure out how to add support for explicitly pinning user mode processes to the cores. The ideal configuration would be to have everything run only on the X100s, but with an opt-in mechanism to run a program only on an A100 core.