9 ms·

> "[CPU, Memory, Disk] is cheap, engineering time isn't." Fuck that. It is. It's absurdly cheap. I ensure I check the amount of time it would take for me to

> "[CPU, Memory, Disk] is cheap, engineering time isn't." Fuck that. It is. It's absurdly cheap. I ensure I check the amount of time it would take for me to make a performance improvement against the runtime costs of my functions. It's rarely worth the extra effort. Seriously, until you get into the millions of records per second level, you're almost never benefited. You may make your function 2x faster, at a cost of additional complexity, but you never run it enough in a year for it to pay itself back. > Bad engineering time is _incredibly_ expensive. Engineering time is expensive. Period. It speaks to the need to minimize it. > This is an excuse to not spend time learning the ins and outs of your language, and your hardware. All of which will change in a few years, which is fine, if you're also committing to keeping _all that code_ up to date right along with it. Otherwise you end up with an obscure mess that you have to unwind 5 years of context to understand and fix again. Complexity and available mental contexts are forgotten costs. If your language even has that many "ins and outs" to begin with you may want to reconsider that.

sgarland 2y ago
> You may make your function 2x faster, at a cost of additional complexity, but you never run it enough in a year for it to pay itself back. I'm not talking about increased complexity, I'm talking about extremely basic things that take zero extra time, like using the correct data structure. For example, in Python: In [8]: a = array("i", (x for x in range(1_000_000))) ...: l = [x for x in range(1_000_000)] ...: d = deque(l) ...: for x in (a, l, d): ...: print(f"{sys.getsizeof(x) / 2**20} MiB") ...: 3.902385711669922 MiB 8.057334899902344 MiB 7.868537902832031 MiB Very similar structures, with very different memory requirements and access speeds. I can count on one hand with no fingers the number of times I've seen an array used. Or knowing that `random.randint` is remarkably slow compared to `random.random()`, which can matter in a hot loop: In [10]: %timeit math.floor(random.random() * 1_000_000) 31.9 ns ± 0.138 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) In [11]: %timeit random.randint(0, 1_000_000) 163 ns ± 0.653 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) > All of which will change in a few years, which is fine, if you're also committing to keeping _all that code_ up to date right along with it. With the exception of list comprehension over large ranges slowing down from 3.11 --> now, I don't think there's been much in Python that's become dramatically worse such that you would need to refactor it later (I gather the Javascript community does this ritual every quarter or so). Anything being deprecated has years of warning.
- akira2501 2y ago
  > which can matter in a hot loop: 163ns - 31.9ns == 131.1ns This will need to happen 7.6 million times to save me 1 CPU second. On AWS lambda with 1GB of memory this will cost you a whopping: $0.0000166667. The point is, you're not even wrong, but there are vanishingly few cases where it would actually matter to the bottom line in practice. You're taking an absolutist point of view to a discipline which thoroughly rejects it. This is what I love about the cloud. It forces you to confront what your efforts are actually worth by placing a specific value on all of these commodities. In my experience they're often worth very little given that none of us have the scale of problems where this would show actual returns.
  - norir 2y ago
    Sure, but the cumulative effects of pervasive mediocre to bad decisions do add up. And it isn't just about cloud compute cost. Your own time is stolen by the slow ci jobs that you inevitably get stuck waiting for. For me, I prioritize my own personal happiness in my work and this mindset taken too far makes me unhappy.
  - zrm 2y ago
    Reaching the scale where it shows actual returns isn't all that difficult. You need it to happen 7.6 million times to save 1 CPU second, but each CPU core can execute it nearly that many times every second. Probably you don't leave it generating only random numbers all day, but suppose you do generate a good few, so that it's 1% of your total compute budget, and you have only a modest load, using on average four CPU cores at any given time. Then saving that amount of computation will have saved you something like $15/year in compute, recurring. Which isn't actually that bad a return for ten seconds worth of choosing the right function. There are also a lot of even fairly small entities for which four cores is peanuts and they're running a hundred or a thousand at once, which quickly turns up the price. And even the things with internet scale aren't all that rare. Suppose you're making a contribution to the mainline Linux kernel. It will run on billions of devices, possibly for decades. Even if it doesn't run very often, that's still a lot of cycles, and some of the kernel code does run very often. Likewise code in popular web browsers, javascript on popular websites or in popular libraries, etc. You don't have to work for Google to make a contribution to zlib and that kind of stuff has the weight of the world on it.
jjav 2y ago
> You may make your function 2x faster, at a cost of additional complexity, but you never run it enough in a year for it to pay itself back. "You" is both singular and plural, which is often the problem with this thinking. Is it worth spending a month of engineering time to make a page load in 50ms instead of 2s? Seems like a lot of engineering time for a noticeable but somewhat minor improvement. But now, what if you have a million users who do this operation 100x/day? Absolutely worth it! For example, I sure wish atlassian would spend a tiny bit of effort into making jira faster. Even if it is 1 second per ticket, since I'm viewing 100+ tickets per day that adds up. And there's many hundreds of us at the company doing the same thing, it really adds up.
- ddtaylor 2y ago
  > 50ms instead of 2s In the past I believe Google was very adament that page load time perception was very important to other metrics.
- sfn42 2y ago
  You're probably not going to achieve that with the kind of optimization described in this article though.
- nottorp 2y ago
  Most of the time they just move the expensive processing to the user's browser so they don't have to pay for it :)
- xlii 2y ago
  Nit: 50ms vs 2000ms is 40x speed increase, i.e. ~1.5 order of magnitude. I still keep words of my database optimization lecturer who said that by his experience optimization below 1 OOM aren’t worth it and most „good ones” are 3+ > Absolutely worth it! Long reaching assumption. Even the biggest companies have limited resources (even if vast). Would you rather improve load times by 2x (from 500ms to 250ms) or improve checkout reliability from 99% to 99.5%? And there is much more to consider on some levels (e.g. planning for thermal efficiency is fun). Software development is always a game of choice.
Const-me 2y ago
> until you get into the millions of records per second level, you're almost never benefited Yeah, but the software landscape is very diverse. On my job (CAM/CAE) I often handle data structures with gigabytes of data. Worse, unlike e.g. multimedia frameworks many algorithms operating on these numbers are global i.e. can’t be represented as a pipeline which splits data into independent chunks and processes them sequentially. Making performance critical functions twice as fast might saves hours of time for a single end user in a single day.

sgarland 2y ago

> You may make your function 2x faster, at a cost of additional complexity, but you never run it enough in a year for it to pay itself back. I'm not talking about increased complexity, I'm talking about extremely basic things that take zero extra time, like using the correct data structure. For example, in Python: In [8]: a = array("i", (x for x in range(1_000_000))) ...: l = [x for x in range(1_000_000)] ...: d = deque(l) ...: for x in (a, l, d): ...: print(f"{sys.getsizeof(x) / 2**20} MiB") ...: 3.902385711669922 MiB 8.057334899902344 MiB 7.868537902832031 MiB Very similar structures, with very different memory requirements and access speeds. I can count on one hand with no fingers the number of times I've seen an array used. Or knowing that `random.randint` is remarkably slow compared to `random.random()`, which can matter in a hot loop: In [10]: %timeit math.floor(random.random() * 1_000_000) 31.9 ns ± 0.138 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) In [11]: %timeit random.randint(0, 1_000_000) 163 ns ± 0.653 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) > All of which will change in a few years, which is fine, if you're also committing to keeping _all that code_ up to date right along with it. With the exception of list comprehension over large ranges slowing down from 3.11 --> now, I don't think there's been much in Python that's become dramatically worse such that you would need to refactor it later (I gather the Javascript community does this ritual every quarter or so). Anything being deprecated has years of warning.

akira2501 2y ago
> which can matter in a hot loop: 163ns - 31.9ns == 131.1ns This will need to happen 7.6 million times to save me 1 CPU second. On AWS lambda with 1GB of memory this will cost you a whopping: $0.0000166667. The point is, you're not even wrong, but there are vanishingly few cases where it would actually matter to the bottom line in practice. You're taking an absolutist point of view to a discipline which thoroughly rejects it. This is what I love about the cloud. It forces you to confront what your efforts are actually worth by placing a specific value on all of these commodities. In my experience they're often worth very little given that none of us have the scale of problems where this would show actual returns.
- norir 2y ago
  Sure, but the cumulative effects of pervasive mediocre to bad decisions do add up. And it isn't just about cloud compute cost. Your own time is stolen by the slow ci jobs that you inevitably get stuck waiting for. For me, I prioritize my own personal happiness in my work and this mindset taken too far makes me unhappy.
- zrm 2y ago
  Reaching the scale where it shows actual returns isn't all that difficult. You need it to happen 7.6 million times to save 1 CPU second, but each CPU core can execute it nearly that many times every second. Probably you don't leave it generating only random numbers all day, but suppose you do generate a good few, so that it's 1% of your total compute budget, and you have only a modest load, using on average four CPU cores at any given time. Then saving that amount of computation will have saved you something like $15/year in compute, recurring. Which isn't actually that bad a return for ten seconds worth of choosing the right function. There are also a lot of even fairly small entities for which four cores is peanuts and they're running a hundred or a thousand at once, which quickly turns up the price. And even the things with internet scale aren't all that rare. Suppose you're making a contribution to the mainline Linux kernel. It will run on billions of devices, possibly for decades. Even if it doesn't run very often, that's still a lot of cycles, and some of the kernel code does run very often. Likewise code in popular web browsers, javascript on popular websites or in popular libraries, etc. You don't have to work for Google to make a contribution to zlib and that kind of stuff has the weight of the world on it.

jjav 2y ago

> You may make your function 2x faster, at a cost of additional complexity, but you never run it enough in a year for it to pay itself back. "You" is both singular and plural, which is often the problem with this thinking. Is it worth spending a month of engineering time to make a page load in 50ms instead of 2s? Seems like a lot of engineering time for a noticeable but somewhat minor improvement. But now, what if you have a million users who do this operation 100x/day? Absolutely worth it! For example, I sure wish atlassian would spend a tiny bit of effort into making jira faster. Even if it is 1 second per ticket, since I'm viewing 100+ tickets per day that adds up. And there's many hundreds of us at the company doing the same thing, it really adds up.

ddtaylor 2y ago
> 50ms instead of 2s In the past I believe Google was very adament that page load time perception was very important to other metrics.
sfn42 2y ago
You're probably not going to achieve that with the kind of optimization described in this article though.
nottorp 2y ago
Most of the time they just move the expensive processing to the user's browser so they don't have to pay for it :)
xlii 2y ago
Nit: 50ms vs 2000ms is 40x speed increase, i.e. ~1.5 order of magnitude. I still keep words of my database optimization lecturer who said that by his experience optimization below 1 OOM aren’t worth it and most „good ones” are 3+ > Absolutely worth it! Long reaching assumption. Even the biggest companies have limited resources (even if vast). Would you rather improve load times by 2x (from 500ms to 250ms) or improve checkout reliability from 99% to 99.5%? And there is much more to consider on some levels (e.g. planning for thermal efficiency is fun). Software development is always a game of choice.

Const-me 2y ago

> until you get into the millions of records per second level, you're almost never benefited Yeah, but the software landscape is very diverse. On my job (CAM/CAE) I often handle data structures with gigabytes of data. Worse, unlike e.g. multimedia frameworks many algorithms operating on these numbers are global i.e. can’t be represented as a pipeline which splits data into independent chunks and processes them sequentially. Making performance critical functions twice as fast might saves hours of time for a single end user in a single day.