5 ms·

Completely agree re: firedancer codebase. There is a level of thought and discipline wrt performance that I have never seen anywhere else.

SEJeff 2y ago
That team is full of world experts in high performance computing.
dhx 2y ago
It's much more than just performance they've thought about. Here are some of the secure programming practices that have been implemented: /* All the functions in this file are considered "secure", specifically: - Constant time in the input, i.e. the input can be a secret[2] - Small and auditable code base, incl. simple types - Either, no local variables = no need to clear them before exit (most functions) - Or, only static allocation + clear local variable before exit (fd_ed25519_scalar_mul_base_const_time) - Clear registers via FD_FN_SENSITIVE[3] - C safety */ libsodium[4] implements similar mechanisms, and Linux kernel encryption code does too (example: use of kfree_sensitive)[5]. However, firedancer appears to better avoid moving secrets outside of CPU registers, and [3] explains that libraries such as libsodium have inadequate zeroisation, something which firedancer claims to improve upon. [1] https://github.com/firedancer-io/firedancer/blob/main/src/ballet/ed25519/fd_curve25519_secure.c https://github.com/firedancer-io/firedancer/blob/main/src/ba... [2] https://en.wikipedia.org/wiki/Elliptic_curve_point_multiplication#Constant_time_Montgomery_ladder https://en.wikipedia.org/wiki/Elliptic_curve_point_multiplic... [3] https://eprint.iacr.org/2023/1713 https://eprint.iacr.org/2023/1713 [4] https://libsodium.gitbook.io/doc/internals#security-first https://libsodium.gitbook.io/doc/internals#security-first [5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/crypto/ecc.c https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- tptacek 2y ago
  These are table stakes for core cryptographic code, and SOT crypto code --- like the Amazon implementation this story is about --- tend at this point all to be derived from formal methods.
  - dhx 2y ago
    As an example, the Amazon implementation doesn't refer to gcc's[1] and clang's[2] "zero_call_used_regs" to zeroise CPU registers upon return or exception of functions working on crypto secrets. OpenSSL doesn't either.[3] firedancer _does_ use "zero_call_used_regs" to allow gcc/clang to zeroise used CPU registers.[9] As another example, the Amazon implementation also doesn't refer to gcc's "strub" attribute which zeroises the function's stack upon return or exception of functions working on crypto secrets.[4][5] OpenSSL doesn't either.[3] firedancer _does_ use the "strub" attribute to allow gcc to zeroise the function's stack.[9] Is there a performance impact? [6] has the overhead at 0% for X25519 for implementing CPU register and stack zeroisation. Compiling the Linux kernel with "CONFIG_ZERO_CALL_USED_REGS=1" for x64_64 (impacting all kernel functions) was found to result in a 1-1.5% performance penalty.[7][8] [1] https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-zero_005fcall_005fused_005fregs-function-attribute https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute... [2] https://clang.llvm.org/docs/AttributeReference.html#zero-call-used-regs https://clang.llvm.org/docs/AttributeReference.html#zero-cal... [3] https://github.com/openssl/openssl/discussions/24321 https://github.com/openssl/openssl/discussions/24321 [4] https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Common-Type-Attributes.html#index-strub-type-attribute https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Common-Type-At... [5] https://gcc.gnu.org/onlinedocs/gcc/Stack-Scrubbing.html https://gcc.gnu.org/onlinedocs/gcc/Stack-Scrubbing.html [6] https://eprint.iacr.org/2023/1713.pdf https://eprint.iacr.org/2023/1713.pdf [7] https://www.phoronix.com/review/zero-used-regs/5 https://www.phoronix.com/review/zero-used-regs/5 [8] https://lore.kernel.org/lkml/20210505191804.4015873-1-keescook@chromium.org/ https://lore.kernel.org/lkml/20210505191804.4015873-1-keesco... [9] FD_FN_UNSANITIZED: https://github.com/firedancer-io/firedancer/blob/master/src/util/fd_util_base.h https://github.com/firedancer-io/firedancer/blob/master/src/...
    - jandrese 2y ago
      Zeroizing a register seems pretty straightforward. Zeroizing any cache that it may have touched seems a lot more complex. I guess that's why they work so hard to keep everything in registers. Lucky for them we aren't in the x86 era anymore and there are a useful number of registers. I'll need to read up on how they avoid context switches while their registers are loaded.

SEJeff 2y ago

That team is full of world experts in high performance computing.

dhx 2y ago

It's much more than just performance they've thought about. Here are some of the secure programming practices that have been implemented: /* All the functions in this file are considered "secure", specifically: - Constant time in the input, i.e. the input can be a secret[2] - Small and auditable code base, incl. simple types - Either, no local variables = no need to clear them before exit (most functions) - Or, only static allocation + clear local variable before exit (fd_ed25519_scalar_mul_base_const_time) - Clear registers via FD_FN_SENSITIVE[3] - C safety */ libsodium[4] implements similar mechanisms, and Linux kernel encryption code does too (example: use of kfree_sensitive)[5]. However, firedancer appears to better avoid moving secrets outside of CPU registers, and [3] explains that libraries such as libsodium have inadequate zeroisation, something which firedancer claims to improve upon. [1] https://github.com/firedancer-io/firedancer/blob/main/src/ballet/ed25519/fd_curve25519_secure.c https://github.com/firedancer-io/firedancer/blob/main/src/ba... [2] https://en.wikipedia.org/wiki/Elliptic_curve_point_multiplication#Constant_time_Montgomery_ladder https://en.wikipedia.org/wiki/Elliptic_curve_point_multiplic... [3] https://eprint.iacr.org/2023/1713 https://eprint.iacr.org/2023/1713 [4] https://libsodium.gitbook.io/doc/internals#security-first https://libsodium.gitbook.io/doc/internals#security-first [5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/crypto/ecc.c https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

tptacek 2y ago
These are table stakes for core cryptographic code, and SOT crypto code --- like the Amazon implementation this story is about --- tend at this point all to be derived from formal methods.
- dhx 2y ago
  As an example, the Amazon implementation doesn't refer to gcc's[1] and clang's[2] "zero_call_used_regs" to zeroise CPU registers upon return or exception of functions working on crypto secrets. OpenSSL doesn't either.[3] firedancer _does_ use "zero_call_used_regs" to allow gcc/clang to zeroise used CPU registers.[9] As another example, the Amazon implementation also doesn't refer to gcc's "strub" attribute which zeroises the function's stack upon return or exception of functions working on crypto secrets.[4][5] OpenSSL doesn't either.[3] firedancer _does_ use the "strub" attribute to allow gcc to zeroise the function's stack.[9] Is there a performance impact? [6] has the overhead at 0% for X25519 for implementing CPU register and stack zeroisation. Compiling the Linux kernel with "CONFIG_ZERO_CALL_USED_REGS=1" for x64_64 (impacting all kernel functions) was found to result in a 1-1.5% performance penalty.[7][8] [1] https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-zero_005fcall_005fused_005fregs-function-attribute https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute... [2] https://clang.llvm.org/docs/AttributeReference.html#zero-call-used-regs https://clang.llvm.org/docs/AttributeReference.html#zero-cal... [3] https://github.com/openssl/openssl/discussions/24321 https://github.com/openssl/openssl/discussions/24321 [4] https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Common-Type-Attributes.html#index-strub-type-attribute https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Common-Type-At... [5] https://gcc.gnu.org/onlinedocs/gcc/Stack-Scrubbing.html https://gcc.gnu.org/onlinedocs/gcc/Stack-Scrubbing.html [6] https://eprint.iacr.org/2023/1713.pdf https://eprint.iacr.org/2023/1713.pdf [7] https://www.phoronix.com/review/zero-used-regs/5 https://www.phoronix.com/review/zero-used-regs/5 [8] https://lore.kernel.org/lkml/20210505191804.4015873-1-keescook@chromium.org/ https://lore.kernel.org/lkml/20210505191804.4015873-1-keesco... [9] FD_FN_UNSANITIZED: https://github.com/firedancer-io/firedancer/blob/master/src/util/fd_util_base.h https://github.com/firedancer-io/firedancer/blob/master/src/...
  - jandrese 2y ago
    Zeroizing a register seems pretty straightforward. Zeroizing any cache that it may have touched seems a lot more complex. I guess that's why they work so hard to keep everything in registers. Lucky for them we aren't in the x86 era anymore and there are a useful number of registers. I'll need to read up on how they avoid context switches while their registers are loaded.