9 ms·

Not sure I follow. Is "fuzzing their Zig code" somehow related to adding a Fuzzilli integration to Bun?

mtlynch 2mo ago
Yes, "their" refers to Bun's code, not the Zig compiler's code. Fuzzili is a fuzzing engine for JavaScript, so integrating it into Bun means that Fuzzili is fuzzing Bun.[0] From the Bun post[1] > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore From Andrew Kelley's post today[2]: > The post claims they were fuzzing their Zig code, while during our calls the whole Bun team told us that they were not fuzzing anything. This appears to be an outright fabrication. Sumner says that the Bun team has been fuzzing Bun's Zig code. Kelley says that this is a fabrication. Sumner showed proof that the Bun team has been fuzzing Bun's Zig code. It looks like Kelley is incorrect and made an unfounded claim. The generous interpretation is that at the time Kelley and Sumner had a more collaborative relationship, Sumner was not fuzzing Bun's Zig code, but I'd expect Kelley to check if anything had changed since then before publicly accusing Sumner of lying in this week's Bun blog post. [0] https://github.com/googleprojectzero/fuzzilli https://github.com/googleprojectzero/fuzzilli [1] https://bun.com/blog/bun-in-rust https://bun.com/blog/bun-in-rust [2] https://andrewkelley.me/post/my-thoughts-bun-rust-rewrite.html https://andrewkelley.me/post/my-thoughts-bun-rust-rewrite.ht...
- orangeisthe 2mo ago
  Why are you not using hyperlinks and adding links citation style?
  - Philpax 2mo ago
    This is standard convention on HN for when you have more than one link. HN does not support masked links.
- xlii 2mo ago
  AFAIU fuzzing code != fuzzing results. Through skimming it seems that integration tests were using fuzzing, but I would call it fuzzing the code itself. From "product" perspective there's no difference, but in program-compiler perspective (and e.g. raising bugs about compiler), Fuzilli isn't fuzzing. Per Wikipedia > (then...) The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. As for myself, I wouldn't use term fuzzing for integration testing such the one used by Fuzilla. I always caught it dynamic testing, scenario testing and in bigger cases property based tests. Fuzzing in my mind is reserved to a low-abstraction calls. Might just be me, though.
  - jsnell 2mo ago
    I don't understand what distinction you're trying to draw here. The very specific claim[0] in the Bun blog post that Kelley is calling a fabrication was: > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore It does not look to be a fabrication, and is very explicit just about what they meant by fuzzing. [0] I mean, that sentence doesn't actually match Kelley's paraphrase, but it is literally the only claim in the post related to what fuzzing was done on the Zig-based bun codebase. So it has to be what Kelley was referring to, and his paraphrase is as sloppy as his fact-checking.
    - xlii 2mo ago
      For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result).
      - mtlynch 2mo ago
        > For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. I've never heard anyone restrict the definition of "fuzzing" in this way. If I repeatedly generate inputs to a program and then run the program with those inputs, that's fuzzing. It doesn't matter if there's IO or not. > Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result). It's still a compiler. It translates code from one language to another. You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler.
- samtheprogram 2mo ago
  Based on timeline, it seems like both are true. They stopped communicating around the time of the acquisition per OP, which was announced December 3rd, and the PR integrating it is was merged the tail end of November.

Yes, "their" refers to Bun's code, not the Zig compiler's code. Fuzzili is a fuzzing engine for JavaScript, so integrating it into Bun means that Fuzzili is fuzzing Bun.[0] From the Bun post[1] > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore From Andrew Kelley's post today[2]: > The post claims they were fuzzing their Zig code, while during our calls the whole Bun team told us that they were not fuzzing anything. This appears to be an outright fabrication. Sumner says that the Bun team has been fuzzing Bun's Zig code. Kelley says that this is a fabrication. Sumner showed proof that the Bun team has been fuzzing Bun's Zig code. It looks like Kelley is incorrect and made an unfounded claim. The generous interpretation is that at the time Kelley and Sumner had a more collaborative relationship, Sumner was not fuzzing Bun's Zig code, but I'd expect Kelley to check if anything had changed since then before publicly accusing Sumner of lying in this week's Bun blog post. [0] https://github.com/googleprojectzero/fuzzilli https://github.com/googleprojectzero/fuzzilli [1] https://bun.com/blog/bun-in-rust https://bun.com/blog/bun-in-rust [2] https://andrewkelley.me/post/my-thoughts-bun-rust-rewrite.html https://andrewkelley.me/post/my-thoughts-bun-rust-rewrite.ht...

orangeisthe 2mo ago
Why are you not using hyperlinks and adding links citation style?
- Philpax 2mo ago
  This is standard convention on HN for when you have more than one link. HN does not support masked links.
xlii 2mo ago
AFAIU fuzzing code != fuzzing results. Through skimming it seems that integration tests were using fuzzing, but I would call it fuzzing the code itself. From "product" perspective there's no difference, but in program-compiler perspective (and e.g. raising bugs about compiler), Fuzilli isn't fuzzing. Per Wikipedia > (then...) The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. As for myself, I wouldn't use term fuzzing for integration testing such the one used by Fuzilla. I always caught it dynamic testing, scenario testing and in bigger cases property based tests. Fuzzing in my mind is reserved to a low-abstraction calls. Might just be me, though.
- jsnell 2mo ago
  I don't understand what distinction you're trying to draw here. The very specific claim[0] in the Bun blog post that Kelley is calling a fabrication was: > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore It does not look to be a fabrication, and is very explicit just about what they meant by fuzzing. [0] I mean, that sentence doesn't actually match Kelley's paraphrase, but it is literally the only claim in the post related to what fuzzing was done on the Zig-based bun codebase. So it has to be what Kelley was referring to, and his paraphrase is as sloppy as his fact-checking.
  - xlii 2mo ago
    For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result).
    - mtlynch 2mo ago
      > For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. I've never heard anyone restrict the definition of "fuzzing" in this way. If I repeatedly generate inputs to a program and then run the program with those inputs, that's fuzzing. It doesn't matter if there's IO or not. > Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result). It's still a compiler. It translates code from one language to another. You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler.
samtheprogram 2mo ago
Based on timeline, it seems like both are true. They stopped communicating around the time of the acquisition per OP, which was announced December 3rd, and the PR integrating it is was merged the tail end of November.