6 ms·

I know the point of this article is to demonstrate how to solve a problem but the premise is a bad one and must not be confused with sound software architecture

by gw98 4y ago

I know the point of this article is to demonstrate how to solve a problem but the premise is a bad one and must not be confused with sound software architecture at all. In fact it's slightly painful reading it and I really wish people would stop writing articles like this. The functional requirement here is to take some HTML, parse it and emit slack flavoured markdown. Involving WASM / Rust / cross compiling stuff and fucking around with a build tool chain should not even be being discussed anywhere as a solution for this. I mean it's fine as a toy but the problem is that half the industry doesn't have any idea what's a good idea and what isn't from an engineering perspective when it comes to solving problems and this will be taken as gospel on how to solve the thing. What we have here is a Rube Goldberg machine, not a cleanly solved engineering problem.

Gordonjcp 4y ago
- deleted 4y ago
  [deleted]
- dom96 4y ago
  > Exactly. What the fuck even is this? There is constructive criticism and then there is this. Once you use phrases like the above and follow it up by pontificating about how you think the author's house looks like, you have crossed a line, by far.
  - nells 4y ago
    Some people deserve to read “what the fuck is this” about what they did.
    - gw98 4y ago
      This. I'd rather have to defend all my ideas than have the bad ones blindly promoted.
      - nequo 4y ago
        There is an important difference between “your idea is misguided for reasons X and Y” and “what the fuck even is this.” Besides, see the guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- ggm 4y ago
  When I could write a single script in awk or Python I do still write cat < file | cut | awk | sed | wc (notice useless use of cat)
  - stepbeek 4y ago
    I know that the use of cat isn't technically required, but I still build bash oneliners step-by-step and find starting with `cat foo` to be a helpful reminder of the format of the file.
zaarn 4y ago
This is a toy example to demonstrate a framework to build with. But I don't think it's not a cleanly solved engineering problem. Instead of reinventing xer Rust code in Go, Xe has simply taken the functional rust code and made go run it in a cross-platform and portable manner. Minimal effort from the engineer has been expended to solve this and C was not involved for FFI definitions either.
- morelisp 4y ago
  > Minimal effort from the engineer has been expended Is the definition of "how do I get promoted in a company where I don't care if it survives the next five years", not "a cleanly solved engineering problem".
  - zaarn 4y ago
    I don't really see the maintenance issue. The rust code seems maintainable and the go code seems to be fairly lightweight overall.
    - gw98 4y ago
      You missed the two entire toolchains and the third tool required to stick them together? It's pretty difficult and expensive to build one stack, let alone three and onboard another tool.
      - dom96 4y ago
        Depends on how well supported those stacks are. WASM is very well supported and likely to be getting tested/used/improved extensively as the years go. I'd rather work on software that depends on three different tech stacks that are well understood and used by many, than software that depends on a single niche tech stack.
        morelisp 4y ago
        I'm not a "stick to a single stack for everything" kind of person, but here we're comparing Go or Rust, to Go + Rust + WASM. The first option is strictly and substantially less risky in this dimension.
morelisp 4y ago
It's the flattening of everything. For a system architect or senior developer it's very interesting to know I might glue a (not so trivial) Rust codebase into my Go program so easily. For people actually working on Go, Rust, WASM, etc., these are also experiments to evaluate the ergonomics, performance, etc. of all the tooling. For someone who wants to learn how FFIs work, this is a great tutorial. But I'm certain I will now get at least one mid-level dev or interview candidate actually try to do this, in a similarly trivial case. And I will have to explain yes you already wrote the whole build pipeline but no we're not going to maintain it, and yes the blog post says it's fast but it's not really that fast, and etc. etc. It's bad enough every time I have to hear "it's just one Python script, what could it cost?", but the more complex it gets the more energy it takes to genuinely convince (rather than merely authoritatively declare) people not to do it, and the limit there seems unbounded.
kibwen 4y ago
This appears to be a straw man. Nobody is trying to tell you to rewrite your software stack using this technique. The OP demonstrates a cool hack. The site we currently occupying is a place for cool hacks. I don't see the problem. As far as hacks go, it's far from the most egregious that I've seen, and even suggests a few thoughtful lessons about the future of FFI beyond the C ABI.
mediocregopher 4y ago
It's very clearly a toy example to demonstrate the idea, and it does so well. > What we have here is a Rube Goldberg machine, not a cleanly solved engineering problem. And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem. People have been using linked libraries to re-use code across languages since forever, it's fine. This solution isn't very different, it just makes shipping easier. As a parting comment, "sound software architecture" is not a decided upon principle which can be empirically determined, and if it was we'd all be out of a job.
- morelisp 4y ago
  > People have been using linked libraries to re-use code across languages since forever, it's a fine way to solve problems. People have been doing it forever, but people have also hated it forever. Twenty years ago you saw SWIG in a build, you'd know you were probably in for a bad time in an exciting new way.
  - mk89 4y ago
    I think SWIG had a different purpose. The author here generates and embeds WebAssembly in Go, to avoid building the same lib for multiple platforms (+ the bindings to call the low level c code). Maybe the tool wasn't good enough? Right now this is just WebAssembly which is proven to work on multiple platforms. If the API is clear and documented, I don't see why this would be an issue, except for the fact it might be a little bit clunky. It's not the first solution I would come up with, but the question would be: why not? Just because we're used to older and more traditional patterns, why not just to embed webassembly for low level stuff in your code?
- gw98 4y ago
  > And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem Absolutely no way is this a fine way to solve the problem. That is crazy talk. 1. It introduces additional toolchains into the solution when it is unnecessary. 2. It now means you need multiple language specialists to maintain it and associated communications and context switching. 3. More interfaces and integration means more fragility when it comes to debugging, problem solving as well as increasing the problem surface. 4. It massively increases the dependency stack which means there are now multiple sets of supply chain issues and patching required. This makes no problems easier at all! It's even a bad last resort if that's all you have left to try. Sound software architecture is very very well defined and this is definitely not it. I have seen entire companies burn by taking this approach to problem solving. I'm really getting tired of solutions before problems and this is a fundamental example of it. Give us a real use case not manufacture a problem for it.
xena 4y ago
The article is tagged cursed for a reason. This is not intended to be a good solution, it is intended to be _a solution_ that works better than you'd expect while also explaining the Unix philosophy and going through my entire thought process into making it. The best kinds of hacks are the ones that look ludicrous but are actually somewhat fine in practice.
ilyt 4y ago
> The functional requirement here is to take some HTML, parse it and emit slack flavoured markdown. That is solved already and not what the article is about The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature. That is what article is about. The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting.
- mk89 4y ago
  > The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature. Let me add more to this: speed. You have Rust/WebAsssembly in the mix and automatically you gain in speed as shown in the article. I honestly don't understand the negative comments. If you build a lib in C and then you use standard binding mechanisms, oh it's ok. In this case you leverage a great tool (cargo) to do the heavy lifting for you and bam you get a safe binary that gives you better performance and overall better tooling -> it's bad.... why? I read the original version of the poster's comment and it was way more aggressive and more about "sound architecture". Maybe he/she can explain what exactly is wrong with this approach and what other approach should be taken instead? > The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting. Why? In this case it's a trivial function. What if it's a strong algorithm that needs speed? Rust helps there, and it's faster than Go.
  - masklinn 4y ago
    > Why? Because integrating anything in Go is such a pain in the ass gophers rewrite everything. Thus it makes perfect sense to a gopher.

by gw98 4y ago

Gordonjcp 4y ago

deleted 4y ago
[deleted]
dom96 4y ago
> Exactly. What the fuck even is this? There is constructive criticism and then there is this. Once you use phrases like the above and follow it up by pontificating about how you think the author's house looks like, you have crossed a line, by far.
- nells 4y ago
  Some people deserve to read “what the fuck is this” about what they did.
  - gw98 4y ago
    This. I'd rather have to defend all my ideas than have the bad ones blindly promoted.
    - nequo 4y ago
      There is an important difference between “your idea is misguided for reasons X and Y” and “what the fuck even is this.” Besides, see the guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
ggm 4y ago
When I could write a single script in awk or Python I do still write cat < file | cut | awk | sed | wc (notice useless use of cat)
- stepbeek 4y ago
  I know that the use of cat isn't technically required, but I still build bash oneliners step-by-step and find starting with `cat foo` to be a helpful reminder of the format of the file.

zaarn 4y ago

This is a toy example to demonstrate a framework to build with. But I don't think it's not a cleanly solved engineering problem. Instead of reinventing xer Rust code in Go, Xe has simply taken the functional rust code and made go run it in a cross-platform and portable manner. Minimal effort from the engineer has been expended to solve this and C was not involved for FFI definitions either.

morelisp 4y ago
> Minimal effort from the engineer has been expended Is the definition of "how do I get promoted in a company where I don't care if it survives the next five years", not "a cleanly solved engineering problem".
- zaarn 4y ago
  I don't really see the maintenance issue. The rust code seems maintainable and the go code seems to be fairly lightweight overall.
  - gw98 4y ago
    You missed the two entire toolchains and the third tool required to stick them together? It's pretty difficult and expensive to build one stack, let alone three and onboard another tool.
    - dom96 4y ago
      Depends on how well supported those stacks are. WASM is very well supported and likely to be getting tested/used/improved extensively as the years go. I'd rather work on software that depends on three different tech stacks that are well understood and used by many, than software that depends on a single niche tech stack.
      - morelisp 4y ago
        I'm not a "stick to a single stack for everything" kind of person, but here we're comparing Go or Rust, to Go + Rust + WASM. The first option is strictly and substantially less risky in this dimension.

morelisp 4y ago

It's the flattening of everything. For a system architect or senior developer it's very interesting to know I might glue a (not so trivial) Rust codebase into my Go program so easily. For people actually working on Go, Rust, WASM, etc., these are also experiments to evaluate the ergonomics, performance, etc. of all the tooling. For someone who wants to learn how FFIs work, this is a great tutorial. But I'm certain I will now get at least one mid-level dev or interview candidate actually try to do this, in a similarly trivial case. And I will have to explain yes you already wrote the whole build pipeline but no we're not going to maintain it, and yes the blog post says it's fast but it's not really that fast, and etc. etc. It's bad enough every time I have to hear "it's just one Python script, what could it cost?", but the more complex it gets the more energy it takes to genuinely convince (rather than merely authoritatively declare) people not to do it, and the limit there seems unbounded.

kibwen 4y ago

This appears to be a straw man. Nobody is trying to tell you to rewrite your software stack using this technique. The OP demonstrates a cool hack. The site we currently occupying is a place for cool hacks. I don't see the problem. As far as hacks go, it's far from the most egregious that I've seen, and even suggests a few thoughtful lessons about the future of FFI beyond the C ABI.

mediocregopher 4y ago

It's very clearly a toy example to demonstrate the idea, and it does so well. > What we have here is a Rube Goldberg machine, not a cleanly solved engineering problem. And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem. People have been using linked libraries to re-use code across languages since forever, it's fine. This solution isn't very different, it just makes shipping easier. As a parting comment, "sound software architecture" is not a decided upon principle which can be empirically determined, and if it was we'd all be out of a job.

morelisp 4y ago
> People have been using linked libraries to re-use code across languages since forever, it's a fine way to solve problems. People have been doing it forever, but people have also hated it forever. Twenty years ago you saw SWIG in a build, you'd know you were probably in for a bad time in an exciting new way.
- mk89 4y ago
  I think SWIG had a different purpose. The author here generates and embeds WebAssembly in Go, to avoid building the same lib for multiple platforms (+ the bindings to call the low level c code). Maybe the tool wasn't good enough? Right now this is just WebAssembly which is proven to work on multiple platforms. If the API is clear and documented, I don't see why this would be an issue, except for the fact it might be a little bit clunky. It's not the first solution I would come up with, but the question would be: why not? Just because we're used to older and more traditional patterns, why not just to embed webassembly for low level stuff in your code?
gw98 4y ago
> And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem Absolutely no way is this a fine way to solve the problem. That is crazy talk. 1. It introduces additional toolchains into the solution when it is unnecessary. 2. It now means you need multiple language specialists to maintain it and associated communications and context switching. 3. More interfaces and integration means more fragility when it comes to debugging, problem solving as well as increasing the problem surface. 4. It massively increases the dependency stack which means there are now multiple sets of supply chain issues and patching required. This makes no problems easier at all! It's even a bad last resort if that's all you have left to try. Sound software architecture is very very well defined and this is definitely not it. I have seen entire companies burn by taking this approach to problem solving. I'm really getting tired of solutions before problems and this is a fundamental example of it. Give us a real use case not manufacture a problem for it.

xena 4y ago

The article is tagged cursed for a reason. This is not intended to be a good solution, it is intended to be _a solution_ that works better than you'd expect while also explaining the Unix philosophy and going through my entire thought process into making it. The best kinds of hacks are the ones that look ludicrous but are actually somewhat fine in practice.

ilyt 4y ago

> The functional requirement here is to take some HTML, parse it and emit slack flavoured markdown. That is solved already and not what the article is about The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature. That is what article is about. The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting.

mk89 4y ago
> The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature. Let me add more to this: speed. You have Rust/WebAsssembly in the mix and automatically you gain in speed as shown in the article. I honestly don't understand the negative comments. If you build a lib in C and then you use standard binding mechanisms, oh it's ok. In this case you leverage a great tool (cargo) to do the heavy lifting for you and bam you get a safe binary that gives you better performance and overall better tooling -> it's bad.... why? I read the original version of the poster's comment and it was way more aggressive and more about "sound architecture". Maybe he/she can explain what exactly is wrong with this approach and what other approach should be taken instead? > The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting. Why? In this case it's a trivial function. What if it's a strong algorithm that needs speed? Rust helps there, and it's faster than Go.
- masklinn 4y ago
  > Why? Because integrating anything in Go is such a pain in the ass gophers rewrite everything. Thus it makes perfect sense to a gopher.