5 ms·

> not the best design [...] So those engineers at Shopify worked hard for months on a more performant system, but they missed the obvious structure? They chose

> not the best design [...] So those engineers at Shopify worked hard for months on a more performant system, but they missed the obvious structure? They chose a complex denormalization for no good reason? It may be true, but I think it's presumptuous to belittle their work when we have only partial information. My guess is that they had good reasons to think that the more obvious ways would not scale. And from reading your comments in this thread, I believe your structure would fail at their scale. A SQL query that uses 2 sub-queries with "group by" is probably too heavy. From the post, at peaks there would be millions of active shopping carts. BTW, I suspect most orders are just for 1 or 2 of each item, so the denormalization is not as heavy as it seems.

bijowo1676 1mo ago
i also work in big tech and know that a lot of bullshit design creeps into system design and prod, because everyone is overworked, overstressed, wants to just get things done for the quarterly performance review as to not get shitcanned with severance re concurrency, it is not a big issue at all. stock exchanges deal with HFT traders and can easily deal with concurrency of orders. Same can be implemented with shopify, but I doubt they face the same level of concurrency as stock exchange anywhere near
- soontimes 1mo ago
  > re concurrency, it is not a big issue at all I would really appreciate it if you could write this up as an article. It would be an extremely interesting and valuable read
  - bijowo1676 1mo ago
    Martin Fowler's overview of LMAX Disruptor is one of the best reads on this subject re high-load system design https://martinfowler.com/articles/lmax.html https://martinfowler.com/articles/lmax.html
    - soontimes 1mo ago
      > re concurrency, it is not a big issue at all > LMAX Disruptor If anything this article shows that concurrency is a big issue. It is such a big issue you have to write in-memory single threaded processor with custom journaling. If you have established workflows with MySQL and a team knowing how to work with it, throwing all that to do LMAX is not cost efficient. While there are domains where such approach is suitable and even required due to strict transaction ordering, Shopify case doesn’t look like one of them.
- JamesSwift 1mo ago
  Famously, stock is settled on a delay (and generally doesnt involve physical products that are not fungible). Im sure theres a lot to glean from how they handle concurrency but Im not sure they are solving the same problems.
  - bijowo1676 1mo ago
    settlement is a different process, what exchanges are doing is they match Buy and Sell orders. you have an open Sell 1 APPL for $100.0. Millions of other HFT orders rush to scalp your single order. How do you think exchange matches your Sell to HFT's Buy orders? which Buy order gets fulfilled first?
post_below 1mo ago
You might not have noticed that essentially the entire blog post was AI written. There's even this bit where they discover a remarkable trick: > Each round trip to the database has a cost. For carts with multiple line items, we batch reservation queries using UNION ALL so we fetch all needed units in one round trip Insights like that really don't read like senior level output, and of course, it's LLM output. I'm not sure it's presumptuous to question it.
- Chyzwar 1mo ago
  There is now new type of comment in HN, if you disagree with article you attack the fact that ai was used in post editorial process. People here now dismiss anything that have em dash.
  - maxrev17 1mo ago
    Yeah ai doesn’t mean bad content. It does make for unreadable and unbearable articles though
  - post_below 1mo ago
    You make a lot of assumptions here... that I disagree with something about the article (aside from with its quality) and post rationalized somehow, that it has anything to do with em-dashes, that they just used AI for "post editorial process". None of that is true. What is there to disagree with? It's not making a statement or taking a stance, it's a technical writeup and I included a technical quote from the article which would be weird whether or not it was AI written. It's not post edited by AI, it's written whole cloth by AI. No human cared enough to write it. Possibly if it was better quality you could make the case that's ok, but I suspect that you didn't read the article at all.

bijowo1676 1mo ago

i also work in big tech and know that a lot of bullshit design creeps into system design and prod, because everyone is overworked, overstressed, wants to just get things done for the quarterly performance review as to not get shitcanned with severance re concurrency, it is not a big issue at all. stock exchanges deal with HFT traders and can easily deal with concurrency of orders. Same can be implemented with shopify, but I doubt they face the same level of concurrency as stock exchange anywhere near

soontimes 1mo ago
> re concurrency, it is not a big issue at all I would really appreciate it if you could write this up as an article. It would be an extremely interesting and valuable read
- bijowo1676 1mo ago
  Martin Fowler's overview of LMAX Disruptor is one of the best reads on this subject re high-load system design https://martinfowler.com/articles/lmax.html https://martinfowler.com/articles/lmax.html
  - soontimes 1mo ago
    > re concurrency, it is not a big issue at all > LMAX Disruptor If anything this article shows that concurrency is a big issue. It is such a big issue you have to write in-memory single threaded processor with custom journaling. If you have established workflows with MySQL and a team knowing how to work with it, throwing all that to do LMAX is not cost efficient. While there are domains where such approach is suitable and even required due to strict transaction ordering, Shopify case doesn’t look like one of them.
JamesSwift 1mo ago
Famously, stock is settled on a delay (and generally doesnt involve physical products that are not fungible). Im sure theres a lot to glean from how they handle concurrency but Im not sure they are solving the same problems.
- bijowo1676 1mo ago
  settlement is a different process, what exchanges are doing is they match Buy and Sell orders. you have an open Sell 1 APPL for $100.0. Millions of other HFT orders rush to scalp your single order. How do you think exchange matches your Sell to HFT's Buy orders? which Buy order gets fulfilled first?

post_below 1mo ago

You might not have noticed that essentially the entire blog post was AI written. There's even this bit where they discover a remarkable trick: > Each round trip to the database has a cost. For carts with multiple line items, we batch reservation queries using UNION ALL so we fetch all needed units in one round trip Insights like that really don't read like senior level output, and of course, it's LLM output. I'm not sure it's presumptuous to question it.

Chyzwar 1mo ago
There is now new type of comment in HN, if you disagree with article you attack the fact that ai was used in post editorial process. People here now dismiss anything that have em dash.
- maxrev17 1mo ago
  Yeah ai doesn’t mean bad content. It does make for unreadable and unbearable articles though
- post_below 1mo ago
  You make a lot of assumptions here... that I disagree with something about the article (aside from with its quality) and post rationalized somehow, that it has anything to do with em-dashes, that they just used AI for "post editorial process". None of that is true. What is there to disagree with? It's not making a statement or taking a stance, it's a technical writeup and I included a technical quote from the article which would be weird whether or not it was AI written. It's not post edited by AI, it's written whole cloth by AI. No human cared enough to write it. Possibly if it was better quality you could make the case that's ok, but I suspect that you didn't read the article at all.